From 6c9558f7c2c4fa4313f3a96912c1cad1e5732b98 Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Sat, 13 Feb 2016 03:07:37 -0800 Subject: [PATCH] Various searches and replaces. Movement of TOC association to recursive loop function tocsearch.html --- _data/v1_1/globals.yml | 5 + _data/v1_1/guides.yml | 130 +- _data/v1_1/reference.yml | 88 +- _data/v1_1/samples.yml | 42 +- _data/v1_1/support.yml | 38 +- _includes/pagetoc.html | 1 + _includes/tocsearch.html | 1 + _includes/tree.html | 2 +- _layouts/docwithnav.html | 4 +- v1.1/api-ref.md | 4 +- v1.1/app-admin-detailed.md | 2 - v1.1/basicstutorials.md | 2 - v1.1/deploy-clusters.md | 2 - v1.1/docs/README.md | 18 +- v1.1/docs/admin/README.md | 56 +- v1.1/docs/admin/accessing-the-api.md | 14 +- v1.1/docs/admin/admission-controllers.md | 42 +- v1.1/docs/admin/authentication.md | 4 - v1.1/docs/admin/authorization.md | 23 +- v1.1/docs/admin/cluster-components.md | 20 +- v1.1/docs/admin/cluster-large.md | 137 +- v1.1/docs/admin/cluster-management.md | 40 +- v1.1/docs/admin/cluster-troubleshooting.md | 16 +- v1.1/docs/admin/daemons.md | 44 +- v1.1/docs/admin/dns.md | 4 - v1.1/docs/admin/etcd.md | 10 +- v1.1/docs/admin/garbage-collection.md | 1 - v1.1/docs/admin/high-availability.md | 469 +++-- v1.1/docs/admin/index.md | 56 +- v1.1/docs/admin/introduction.md | 39 +- v1.1/docs/admin/kube-apiserver.md | 88 - v1.1/docs/admin/kube-controller-manager.md | 75 - v1.1/docs/admin/kube-proxy.md | 53 - v1.1/docs/admin/kube-scheduler.md | 48 - v1.1/docs/admin/kubelet.md | 115 -- v1.1/docs/admin/limitrange/README.md | 424 ++--- v1.1/docs/admin/limitrange/index.md | 424 ++--- v1.1/docs/admin/multi-cluster.md | 5 +- v1.1/docs/admin/namespaces.md | 38 +- v1.1/docs/admin/namespaces/README.md | 533 +++--- v1.1/docs/admin/namespaces/index.md | 70 +- v1.1/docs/admin/networking.md | 46 +- v1.1/docs/admin/node.md | 44 +- v1.1/docs/admin/ovs-networking.md | 4 - v1.1/docs/admin/resource-quota.md | 16 +- v1.1/docs/admin/resourcequota/README.md | 348 ++-- v1.1/docs/admin/resourcequota/index.md | 348 ++-- v1.1/docs/admin/salt.md | 207 +-- v1.1/docs/admin/service-accounts-admin.md | 20 +- v1.1/docs/admin/static-pods.md | 281 ++- v1.1/docs/api.md | 20 +- v1.1/docs/design/README.md | 8 +- v1.1/docs/design/access.md | 11 +- v1.1/docs/design/admission_control.md | 12 +- .../design/admission_control_limit_range.md | 24 +- .../admission_control_resource_quota.md | 18 +- v1.1/docs/design/architecture.md | 14 +- v1.1/docs/design/clustering.md | 5 - v1.1/docs/design/clustering/README.md | 72 +- v1.1/docs/design/clustering/index.md | 72 +- .../command_execution_port_forwarding.md | 4 - v1.1/docs/design/daemon.md | 256 ++- v1.1/docs/design/event_compression.md | 8 +- v1.1/docs/design/expansion.md | 28 +- v1.1/docs/design/extending-api.md | 36 +- v1.1/docs/design/horizontal-pod-autoscaler.md | 20 +- v1.1/docs/design/identifiers.md | 4 - v1.1/docs/design/index.md | 8 +- v1.1/docs/design/namespaces.md | 38 +- v1.1/docs/design/networking.md | 14 +- v1.1/docs/design/persistent-storage.md | 32 +- v1.1/docs/design/principles.md | 6 +- v1.1/docs/design/resources.md | 28 +- v1.1/docs/design/secrets.md | 40 +- v1.1/docs/design/security.md | 16 +- v1.1/docs/design/security_context.md | 16 +- v1.1/docs/design/service_accounts.md | 14 +- v1.1/docs/design/simple-rolling-update.md | 10 +- v1.1/docs/design/versioning.md | 4 - v1.1/docs/devel/README.md | 48 +- v1.1/docs/devel/api-conventions.md | 1382 +++++++------- v1.1/docs/devel/api_changes.md | 62 +- v1.1/docs/devel/automation.md | 8 +- v1.1/docs/devel/cherry-picks.md | 8 +- v1.1/docs/devel/cli-roadmap.md | 4 - v1.1/docs/devel/client-libraries.md | 4 - v1.1/docs/devel/coding-conventions.md | 12 +- v1.1/docs/devel/collab.md | 4 - v1.1/docs/devel/developer-guides/vagrant.md | 138 +- v1.1/docs/devel/development.md | 108 +- v1.1/docs/devel/e2e-tests.md | 18 +- v1.1/docs/devel/faster_reviews.md | 12 +- v1.1/docs/devel/flaky-tests.md | 20 +- v1.1/docs/devel/getting-builds.md | 20 +- v1.1/docs/devel/index.md | 48 +- v1.1/docs/devel/kubectl-conventions.md | 10 +- v1.1/docs/devel/making-release-notes.md | 8 +- v1.1/docs/devel/profiling.md | 16 +- v1.1/docs/devel/pull-requests.md | 2 +- v1.1/docs/devel/releasing.md | 24 +- v1.1/docs/devel/scheduler.md | 4 - v1.1/docs/devel/scheduler_algorithm.md | 10 +- .../devel/writing-a-getting-started-guide.md | 8 +- v1.1/docs/getting-started-guides/README.md | 138 +- v1.1/docs/getting-started-guides/aws.md | 229 ++- v1.1/docs/getting-started-guides/azure.md | 2 +- .../getting-started-guides/binary_release.md | 8 +- .../centos/centos_manual_config.md | 371 ++-- .../docs/getting-started-guides/cloudstack.md | 7 +- v1.1/docs/getting-started-guides/coreos.md | 15 +- .../coreos/azure/README.md | 487 +++-- .../coreos/azure/index.md | 514 +++--- .../coreos/bare_metal_calico.md | 262 ++- .../coreos/bare_metal_offline.md | 7 +- .../coreos/coreos_multinode_cluster.md | 66 +- v1.1/docs/getting-started-guides/dcos.md | 313 ++-- .../docker-multinode.md | 179 +- .../docker-multinode/deployDNS.md | 16 +- .../docker-multinode/master.md | 66 +- .../docker-multinode/testing.md | 40 +- .../docker-multinode/worker.md | 54 +- v1.1/docs/getting-started-guides/docker.md | 323 ++-- .../fedora/fedora-calico.md | 121 +- .../fedora/fedora_ansible_config.md | 495 +++-- .../fedora/fedora_manual_config.md | 456 +++-- .../fedora/flannel_multi_node_cluster.md | 386 ++-- v1.1/docs/getting-started-guides/gce.md | 455 +++-- v1.1/docs/getting-started-guides/index.md | 160 +- v1.1/docs/getting-started-guides/juju.md | 447 +++-- .../getting-started-guides/libvirt-coreos.md | 621 +++---- v1.1/docs/getting-started-guides/locally.md | 245 ++- .../logging-elasticsearch.md | 52 +- v1.1/docs/getting-started-guides/logging.md | 78 +- .../getting-started-guides/mesos-docker.md | 5 +- v1.1/docs/getting-started-guides/mesos.md | 750 ++++---- v1.1/docs/getting-started-guides/ovirt.md | 3 - v1.1/docs/getting-started-guides/rackspace.md | 5 +- .../docs/getting-started-guides/rkt/README.md | 52 +- v1.1/docs/getting-started-guides/rkt/index.md | 52 +- v1.1/docs/getting-started-guides/scratch.md | 1629 ++++++++--------- .../getting-started-guides/ubuntu-calico.md | 74 +- v1.1/docs/getting-started-guides/ubuntu.md | 78 +- v1.1/docs/getting-started-guides/vagrant.md | 735 ++++---- v1.1/docs/getting-started-guides/vsphere.md | 181 +- v1.1/docs/index.md | 18 +- v1.1/docs/reporting-security-issues.md | 8 +- v1.1/docs/roadmap.md | 4 - v1.1/docs/troubleshooting.md | 12 +- v1.1/docs/user-guide/README.md | 91 +- v1.1/docs/user-guide/accessing-the-cluster.md | 549 +++--- v1.1/docs/user-guide/annotations.md | 10 +- .../user-guide/application-troubleshooting.md | 65 +- v1.1/docs/user-guide/compute-resources.md | 49 +- v1.1/docs/user-guide/config-best-practices.md | 10 +- .../docs/user-guide/configuring-containers.md | 103 +- .../user-guide/connecting-applications.md | 114 +- ...connecting-to-applications-port-forward.md | 105 +- .../connecting-to-applications-proxy.md | 59 +- v1.1/docs/user-guide/container-environment.md | 29 +- v1.1/docs/user-guide/containers.md | 6 +- v1.1/docs/user-guide/debugging-services.md | 152 +- .../docs/user-guide/deploying-applications.md | 70 +- v1.1/docs/user-guide/deployments.md | 96 +- v1.1/docs/user-guide/docker-cli-to-kubectl.md | 122 +- v1.1/docs/user-guide/downward-api.md | 18 +- v1.1/docs/user-guide/downward-api/README.md | 16 +- v1.1/docs/user-guide/downward-api/index.md | 16 +- .../user-guide/downward-api/volume/README.md | 20 +- .../user-guide/downward-api/volume/index.md | 20 +- .../user-guide/environment-guide/README.md | 194 +- .../user-guide/environment-guide/index.md | 194 +- .../user-guide/getting-into-containers.md | 157 +- .../user-guide/horizontal-pod-autoscaler.md | 25 +- .../horizontal-pod-autoscaling/README.md | 68 +- .../horizontal-pod-autoscaling/index.md | 68 +- v1.1/docs/user-guide/identifiers.md | 8 +- v1.1/docs/user-guide/images.md | 46 +- v1.1/docs/user-guide/index.md | 91 +- v1.1/docs/user-guide/ingress.md | 79 +- .../user-guide/introspection-and-debugging.md | 75 +- v1.1/docs/user-guide/jobs.md | 64 +- v1.1/docs/user-guide/jsonpath.md | 8 +- v1.1/docs/user-guide/known-issues.md | 4 - v1.1/docs/user-guide/kubeconfig-file.md | 439 +++-- v1.1/docs/user-guide/kubectl-overview.md | 75 +- v1.1/docs/user-guide/kubectl/.files_generated | 33 - v1.1/docs/user-guide/kubectl/kubectl.md | 86 - .../user-guide/kubectl/kubectl_annotate.md | 104 -- .../kubectl/kubectl_api-versions.md | 57 - v1.1/docs/user-guide/kubectl/kubectl_apply.md | 82 - .../docs/user-guide/kubectl/kubectl_attach.md | 83 - .../user-guide/kubectl/kubectl_autoscale.md | 92 - .../kubectl/kubectl_cluster-info.md | 57 - .../docs/user-guide/kubectl/kubectl_config.md | 77 - .../kubectl/kubectl_config_set-cluster.md | 81 - .../kubectl/kubectl_config_set-context.md | 74 - .../kubectl/kubectl_config_set-credentials.md | 94 - .../user-guide/kubectl/kubectl_config_set.md | 59 - .../kubectl/kubectl_config_unset.md | 58 - .../kubectl/kubectl_config_use-context.md | 57 - .../user-guide/kubectl/kubectl_config_view.md | 88 - .../docs/user-guide/kubectl/kubectl_create.md | 82 - .../docs/user-guide/kubectl/kubectl_delete.md | 104 -- .../user-guide/kubectl/kubectl_describe.md | 104 -- v1.1/docs/user-guide/kubectl/kubectl_edit.md | 99 - v1.1/docs/user-guide/kubectl/kubectl_exec.md | 84 - .../docs/user-guide/kubectl/kubectl_expose.md | 110 -- v1.1/docs/user-guide/kubectl/kubectl_get.md | 118 -- v1.1/docs/user-guide/kubectl/kubectl_label.md | 105 -- v1.1/docs/user-guide/kubectl/kubectl_logs.md | 94 - .../user-guide/kubectl/kubectl_namespace.md | 60 - v1.1/docs/user-guide/kubectl/kubectl_patch.md | 87 - .../kubectl/kubectl_port-forward.md | 84 - v1.1/docs/user-guide/kubectl/kubectl_proxy.md | 106 -- .../user-guide/kubectl/kubectl_replace.md | 96 - .../kubectl/kubectl_rolling-update.md | 104 -- v1.1/docs/user-guide/kubectl/kubectl_run.md | 121 -- v1.1/docs/user-guide/kubectl/kubectl_scale.md | 93 - v1.1/docs/user-guide/kubectl/kubectl_stop.md | 95 - .../user-guide/kubectl/kubectl_version.md | 65 - v1.1/docs/user-guide/labels.md | 69 +- v1.1/docs/user-guide/liveness/README.md | 179 +- v1.1/docs/user-guide/liveness/index.md | 32 +- v1.1/docs/user-guide/logging-demo/README.md | 12 +- v1.1/docs/user-guide/logging-demo/index.md | 12 +- v1.1/docs/user-guide/logging.md | 193 +- v1.1/docs/user-guide/managing-deployments.md | 163 +- v1.1/docs/user-guide/monitoring.md | 17 +- v1.1/docs/user-guide/namespaces.md | 28 +- v1.1/docs/user-guide/node-selection/README.md | 8 +- v1.1/docs/user-guide/node-selection/index.md | 8 +- v1.1/docs/user-guide/overview.md | 16 +- v1.1/docs/user-guide/persistent-volumes.md | 46 +- .../user-guide/persistent-volumes/README.md | 28 +- .../user-guide/persistent-volumes/index.md | 28 +- v1.1/docs/user-guide/pod-states.md | 10 +- v1.1/docs/user-guide/pods.md | 33 +- v1.1/docs/user-guide/prereqs.md | 26 +- v1.1/docs/user-guide/production-pods.md | 104 +- v1.1/docs/user-guide/quick-start.md | 46 +- .../docs/user-guide/replication-controller.md | 30 +- v1.1/docs/user-guide/resourcequota/README.md | 2 +- v1.1/docs/user-guide/resourcequota/index.md | 2 +- v1.1/docs/user-guide/secrets.md | 77 +- v1.1/docs/user-guide/secrets/README.md | 24 +- v1.1/docs/user-guide/secrets/index.md | 24 +- v1.1/docs/user-guide/security-context.md | 9 +- v1.1/docs/user-guide/service-accounts.md | 42 +- v1.1/docs/user-guide/services-firewalls.md | 9 +- v1.1/docs/user-guide/services.md | 68 +- v1.1/docs/user-guide/sharing-clusters.md | 42 +- v1.1/docs/user-guide/simple-nginx.md | 34 +- v1.1/docs/user-guide/simple-yaml.md | 36 +- v1.1/docs/user-guide/ui.md | 101 +- v1.1/docs/user-guide/update-demo/README.md | 280 ++- v1.1/docs/user-guide/update-demo/index.md | 280 ++- v1.1/docs/user-guide/volumes.md | 60 +- v1.1/docs/user-guide/walkthrough/README.md | 385 ++-- v1.1/docs/user-guide/walkthrough/index.md | 72 +- v1.1/docs/user-guide/walkthrough/k8s201.md | 585 +++--- .../docs/user-guide/working-with-resources.md | 14 +- v1.1/docs/whatisk8s.md | 30 +- v1.1/editdocs.md | 3 +- v1.1/gs-custom.md | 39 +- v1.1/index.md | 1 - v1.1/multi-tier.md | 3 - v1.1/reference.md | 1 - v1.1/samples.md | 1 - v1.1/setup-config.md | 3 - v1.1/support.md | 1 - 270 files changed, 10836 insertions(+), 16324 deletions(-) create mode 100644 _data/v1_1/globals.yml create mode 100644 _includes/tocsearch.html delete mode 100644 v1.1/docs/admin/kube-apiserver.md delete mode 100644 v1.1/docs/admin/kube-controller-manager.md delete mode 100644 v1.1/docs/admin/kube-proxy.md delete mode 100644 v1.1/docs/admin/kube-scheduler.md delete mode 100644 v1.1/docs/admin/kubelet.md delete mode 100644 v1.1/docs/user-guide/kubectl/.files_generated delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_annotate.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_api-versions.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_apply.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_attach.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_autoscale.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_cluster-info.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_config.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_config_set-cluster.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_config_set-context.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_config_set-credentials.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_config_set.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_config_unset.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_config_use-context.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_config_view.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_create.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_delete.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_describe.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_edit.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_exec.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_expose.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_get.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_label.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_logs.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_namespace.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_patch.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_port-forward.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_proxy.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_replace.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_rolling-update.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_run.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_scale.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_stop.md delete mode 100644 v1.1/docs/user-guide/kubectl/kubectl_version.md diff --git a/_data/v1_1/globals.yml b/_data/v1_1/globals.yml new file mode 100644 index 0000000000..fd3be92a16 --- /dev/null +++ b/_data/v1_1/globals.yml @@ -0,0 +1,5 @@ +tocs: +- guides +- reference +- samples +- support \ No newline at end of file diff --git a/_data/v1_1/guides.yml b/_data/v1_1/guides.yml index b7747775b8..b2f1fd891c 100644 --- a/_data/v1_1/guides.yml +++ b/_data/v1_1/guides.yml @@ -7,159 +7,159 @@ toc: - title: Quickstarts section: - title: TODO - 5-minute Quickstart - path: /v1.1/docs/hellonode + path: /v1.1/docs/hellonode/ - title: Kubernetes 101 - path: /v1.1/docs/user-guide/walkthrough/README + path: /v1.1/docs/user-guide/walkthrough/README/ - title: Kubernetes 201 - path: /v1.1/docs/user-guide/walkthrough/k8s201 + path: /v1.1/docs/user-guide/walkthrough/k8s201/ - title: Running Kubernetes section: - title: Picking the Right Solution - path: /v1.1/docs/getting-started-guides/README + path: /v1.1/docs/getting-started-guides/README/ - title: Running Kubernetes on Your Local Machine section: - title: Running Kubernetes Locally via Docker - path: /v1.1/docs/getting-started-guides/docker + path: /v1.1/docs/getting-started-guides/docker/ - title: Running Kubernetes Locally via Vagrant - path: /v1.1/docs/getting-started-guides/vagrant + path: /v1.1/docs/getting-started-guides/vagrant/ - title: Running Kubernetes Locally with No VM - path: /v1.1/docs/getting-started-guides/locally + path: /v1.1/docs/getting-started-guides/locally/ - title: Running Kubernetes on Turn-key Cloud Solutions section: - title: Running Kubernetes on Google Container Engine - path: https://cloud.google.com/container-engine/docs/before-you-begin + path: https://cloud.google.com/container-engine/docs/before-you-begin/ - title: Running Kubernetes on Google Compute Engine - path: /v1.1/docs/getting-started-guides/gce + path: /v1.1/docs/getting-started-guides/gce/ - title: Running Kubernetes on AWS EC2 - path: /v1.1/docs/getting-started-guides/aws + path: /v1.1/docs/getting-started-guides/aws/ - title: Running Kubernetes on Azure - path: /v1.1/docs/getting-started-guides/coreos/azure/README + path: /v1.1/docs/getting-started-guides/coreos/azure/README/ - title: Running Kubernetes on Custom Solutions section: - title: Getting Started From Scratch - path: /v1.1/docs/getting-started-guides/scratch + path: /v1.1/docs/getting-started-guides/scratch/ - title: Custom Cloud Solutions section: - title: AWS or GCE on CoreOS - path: /v1.1/docs/getting-started-guides/coreos + path: /v1.1/docs/getting-started-guides/coreos/ - title: AWS or Joyent on Ubuntu - path: /v1.1/docs/getting-started-guides/juju + path: /v1.1/docs/getting-started-guides/juju/ - title: Rackspace on CoreOS - path: /v1.1/docs/getting-started-guides/rackspace + path: /v1.1/docs/getting-started-guides/rackspace/ - title: On-Premise VMs section: - title: Vagrant or VMware - path: /v1.1/docs/getting-started-guides/coreos + path: /v1.1/docs/getting-started-guides/coreos/ - title: Cloudstack - path: /v1.1/docs/getting-started-guides/cloudstack + path: /v1.1/docs/getting-started-guides/cloudstack/ - title: VMWare - path: /v1.1/docs/getting-started-guides/vsphere + path: /v1.1/docs/getting-started-guides/vsphere/ - title: Juju - path: /v1.1/docs/getting-started-guides/juju + path: /v1.1/docs/getting-started-guides/juju/ - title: libvirt on CoreOS - path: /v1.1/docs/getting-started-guides/libvirt-coreos + path: /v1.1/docs/getting-started-guides/libvirt-coreos/ - title: oVirt - path: /v1.1/docs/getting-started-guides/ovirt + path: /v1.1/docs/getting-started-guides/ovirt/ - title: libvirt or KVM - path: /v1.1/docs/getting-started-guides/fedora/flannel_multi_node_cluster + path: /v1.1/docs/getting-started-guides/fedora/flannel_multi_node_cluster/ - title: Bare Metal section: - title: Offline - path: /v1.1/docs/getting-started-guides/coreos/bare_metal_offline + path: /v1.1/docs/getting-started-guides/coreos/bare_metal_offline/ - title: Fedora via Ansible - path: /v1.1/docs/getting-started-guides/fedora/fedora_ansible_config + path: /v1.1/docs/getting-started-guides/fedora/fedora_ansible_config/ - title: Fedora (Single Node) - path: /v1.1/docs/getting-started-guides/fedora/fedora_manual_config + path: /v1.1/docs/getting-started-guides/fedora/fedora_manual_config/ - title: Fedora (Multi Node) - path: /v1.1/docs/getting-started-guides/fedora/flannel_multi_node_cluster + path: /v1.1/docs/getting-started-guides/fedora/flannel_multi_node_cluster/ - title: Centos - path: /v1.1/docs/getting-started-guides/centos/centos_manual_config + path: /v1.1/docs/getting-started-guides/centos/centos_manual_config/ - title: Ubuntu - path: /v1.1/docs/getting-started-guides/ubuntu + path: /v1.1/docs/getting-started-guides/ubuntu/ - title: Docker (Multi Node) - path: /v1.1/docs/getting-started-guides/docker-multinode + path: /v1.1/docs/getting-started-guides/docker-multinode/ - title: Administering Clusters section: - title: Kubernetes Cluster Admin Guide - path: /v1.1/docs/admin/introduction + path: /v1.1/docs/admin/introduction/ - title: Using Multiple Clusters - path: /v1.1/docs/admin/multi-cluster + path: /v1.1/docs/admin/multi-cluster/ - title: Using Large Clusters - path: /v1.1/docs/admin/cluster-large + path: /v1.1/docs/admin/cluster-large/ - title: Building High-Availability Clusters - path: /v1.1/docs/admin/high-availability + path: /v1.1/docs/admin/high-availability/ - title: Accessing Clusters - path: /v1.1/docs/user-guide/accessing-the-cluster + path: /v1.1/docs/user-guide/accessing-the-cluster/ - title: Sharing a Cluster - path: /v1.1/docs/admin/namespaces/README + path: /v1.1/docs/admin/namespaces/README/ - title: Changing Cluster Size - path: https://github.com/kubernetes/kubernetes/wiki/User-FAQ#how-do-i-change-the-size-of-my-cluster + path: https://github.com/kubernetes/kubernetes/wiki/User-FAQ#how-do-i-change-the-size-of-my-cluster/ - title: Creating a Custom Cluster from Scratch - path: /v1.1/docs/getting-started-guides/scratch + path: /v1.1/docs/getting-started-guides/scratch/ - title: Authenticating Across Clusters with kubeconfig - path: /v1.1/docs/user-guide/kubeconfig-file + path: /v1.1/docs/user-guide/kubeconfig-file/ - title: Configuring Garbage Collection - path: /v1.1/docs/admin/garbage-collection + path: /v1.1/docs/admin/garbage-collection/ - title: Configuring Kubernetes with Salt - path: /v1.1/docs/admin/salt + path: /v1.1/docs/admin/salt/ - title: Common Tasks section: - title: Using Nodes - path: /v1.1/docs/admin/node + path: /v1.1/docs/admin/node/ - title: Assigning Pods to Nodes - path: /v1.1/docs/user-guide/node-selection/README + path: /v1.1/docs/user-guide/node-selection/README/ - title: Using Configuration Files - path: /v1.1/docs/user-guide/simple-yaml + path: /v1.1/docs/user-guide/simple-yaml/ - title: Configuring Containers - path: /v1.1/docs/user-guide/configuring-containers + path: /v1.1/docs/user-guide/configuring-containers/ - title: Using Environment Variables - path: /v1.1/docs/user-guide/environment-guide/README + path: /v1.1/docs/user-guide/environment-guide/README/ - title: Managing Compute Resources - path: /v1.1/docs/user-guide/compute-resources + path: /v1.1/docs/user-guide/compute-resources/ - title: Applying Resource Quotas and Limits - path: /v1.1/docs/admin/resourcequota/README + path: /v1.1/docs/admin/resourcequota/README/ - title: Setting Pod CPU and Memory Limits - path: /v1.1/docs/admin/limitrange/README + path: /v1.1/docs/admin/limitrange/README/ - title: Managing Deployments - path: /v1.1/docs/user-guide/managing-deployments + path: /v1.1/docs/user-guide/managing-deployments/ - title: Deploying Applications - path: /v1.1/docs/user-guide/deploying-applications + path: /v1.1/docs/user-guide/deploying-applications/ - title: Launching, Exposing, and Killing Applications - path: /v1.1/docs/user-guide/quick-start + path: /v1.1/docs/user-guide/quick-start/ - title: Connecting Applications - path: /v1.1/docs/user-guide/connecting-applications + path: /v1.1/docs/user-guide/connecting-applications/ - title: Networking in Kubernetes - path: /v1.1/docs/admin/networking + path: /v1.1/docs/admin/networking/ - title: Creating Servers with External IPs - path: /v1.1/examples/simple-nginx + path: /v1.1/examples/simple-nginx/ - title: Setting Up and Configuring DNS - path: /v1.1/examples/cluster-dns/README + path: /v1.1/examples/cluster-dns/README/ - title: Using DNS Pods and Services - path: /v1.1/docs/admin/dns + path: /v1.1/docs/admin/dns/ - title: Working with Containers - path: /v1.1/docs/user-guide/production-pods + path: /v1.1/docs/user-guide/production-pods/ - title: Creating Pods with the Downward API - path: /v1.1/docs/user-guide/downward-api/README + path: /v1.1/docs/user-guide/downward-api/README/ - title: Using Secrets - path: /v1.1/docs/user-guide/secrets/README + path: /v1.1/docs/user-guide/secrets/README/ - title: Using Persistent Volumes - path: /v1.1/docs/user-guide/persistent-volumes/README + path: /v1.1/docs/user-guide/persistent-volumes/README/ - title: Updating Live Pods - path: /v1.1/docs/user-guide/update-demo/README + path: /v1.1/docs/user-guide/update-demo/README/ - title: Testing and Monitoring section: - title: Simulating Large Test Loads - path: /v1.1/examples/k8petstore/README + path: /v1.1/examples/k8petstore/README/ - title: Checking Pod Health - path: /v1.1/docs/user-guide/liveness/README + path: /v1.1/docs/user-guide/liveness/README/ - title: Using Explorer to Examine the Runtime Environment - path: /v1.1/examples/explorer/README + path: /v1.1/examples/explorer/README/ - title: Resource Usage Monitoring - path: /v1.1/docs/user-guide/monitoring \ No newline at end of file + path: /v1.1/docs/user-guide/monitoring/ \ No newline at end of file diff --git a/_data/v1_1/reference.yml b/_data/v1_1/reference.yml index 830ee0be81..86df6f5585 100644 --- a/_data/v1_1/reference.yml +++ b/_data/v1_1/reference.yml @@ -1,107 +1,109 @@ bigheader: "Reference Documentation" abstract: "Design docs, concept definitions, and references for APIs and CLIs." toc: +- title: Reference Documentation + path: /v1.1/reference/ + - title: Kubernetes API - path: /v1.1/reference section: - title: Kubernetes API Overview - path: /v1.1/docs/api + path: /v1.1/docs/api/ - title: Kubernetes API Reference - path: /v1.1/api-ref + path: /v1.1/api-ref/ - title: kubectl section: - title: kubectl Overview - path: /v1.1/docs/user-guide/kubectl-overview + path: /v1.1/docs/user-guide/kubectl-overview/ - title: kubectl for Docker Users - path: /v1.1/reference/docker-cli-to-kubectl + path: /v1.1/reference/docker-cli-to-kubectl/ - title: kubectl Reference (pretend the child nodes are here) - path: /v1.1/docs/user-guide/kubectl/kubectl + path: /v1.1/docs/user-guide/kubectl/kubectl/ - title: JSONpath - path: /v1.1/docs/user-guide/jsonpath + path: /v1.1/docs/user-guide/jsonpath/ - title: kube-apiserver section: - title: Overview - path: /v1.1/docs/admin/kube-apiserver + path: /v1.1/docs/admin/kube-apiserver/ - title: Authorization Plugins - path: /v1.1/docs/admin/authorization + path: /v1.1/docs/admin/authorization/ - title: Authentication - path: /v1.1/docs/admin/authentication + path: /v1.1/docs/admin/authentication/ - title: Accessing the API - path: /v1.1/docs/admin/accessing-the-api + path: /v1.1/docs/admin/accessing-the-api/ - title: Admission Controllers - path: /v1.1/docs/admin/admission-controllers + path: /v1.1/docs/admin/admission-controllers/ - title: Managing Service Accounts - path: /v1.1/docs/admin/service-accounts-admin + path: /v1.1/docs/admin/service-accounts-admin/ - title: kub-scheduler - path: /v1.1/docs/admin/kube-scheduler + path: /v1.1/docs/admin/kube-scheduler/ - title: kubelet - path: /v1.1/docs/admin/kubelet + path: /v1.1/docs/admin/kubelet/ - title: kube-proxy - path: /v1.1/docs/admin/kube-proxy + path: /v1.1/docs/admin/kube-proxy/ - title: etcd - path: /v1.1/docs/admin/etcd + path: /v1.1/docs/admin/etcd/ - title: Concept Definitions section: - title: Container Environment - path: /v1.1/docs/user-guide/container-environment + path: /v1.1/docs/user-guide/container-environment/ - title: Images - path: /v1.1/docs/user-guide/images + path: /v1.1/docs/user-guide/images/ - title: Downward API - path: /v1.1/docs/user-guide/downward-api + path: /v1.1/docs/user-guide/downward-api/ - title: Pods - path: /v1.1/docs/user-guide/pods + path: /v1.1/docs/user-guide/pods/ - title: Labels and Selectors - path: /v1.1/docs/user-guide/labels + path: /v1.1/docs/user-guide/labels/ - title: Replication Controller - path: /v1.1/docs/user-guide/replication-controller + path: /v1.1/docs/user-guide/replication-controller/ - title: Services - path: /v1.1/docs/user-guide/services + path: /v1.1/docs/user-guide/services/ - title: Volumes - path: /v1.1/docs/user-guide/volumes + path: /v1.1/docs/user-guide/volumes/ - title: Persistent Volumes - path: /v1.1/docs/user-guide/persistent-volumes + path: /v1.1/docs/user-guide/persistent-volumes/ - title: Secrets - path: /v1.1/docs/user-guide/secrets + path: /v1.1/docs/user-guide/secrets/ - title: Names - path: /v1.1/docs/user-guide/identifiers + path: /v1.1/docs/user-guide/identifiers/ - title: Namespaces - path: /v1.1/docs/user-guide/namespaces + path: /v1.1/docs/user-guide/namespaces/ - title: Service Accounts - path: /v1.1/docs/user-guide/service-accounts + path: /v1.1/docs/user-guide/service-accounts/ - title: Annotations - path: /v1.1/docs/user-guide/annotations + path: /v1.1/docs/user-guide/annotations/ - title: Daemon Sets - path: /v1.1/docs/admin/daemons + path: /v1.1/docs/admin/daemons/ - title: Deployments - path: /v1.1/docs/user-guide/deployments + path: /v1.1/docs/user-guide/deployments/ - title: Ingress Resources - path: /v1.1/docs/user-guide/ingress + path: /v1.1/docs/user-guide/ingress/ - title: Horizontal Pod Autoscaling - path: /v1.1/docs/user-guide/horizontal-pod-autoscaler + path: /v1.1/docs/user-guide/horizontal-pod-autoscaler/ - title: Jobs - path: /v1.1/docs/user-guide/jobs + path: /v1.1/docs/user-guide/jobs/ - title: Resource Quotas - path: /v1.1/docs/admin/resource-quota + path: /v1.1/docs/admin/resource-quota/ - title: Kubernetes Design Docs section: - title: Kubernetes Architecture - path: /v1.1/docs/design/architecture + path: /v1.1/docs/design/architecture/ - title: Kubernetes Design Overview - path: /v1.1/docs/design/README + path: /v1.1/docs/design/README/ - title: Security in Kubernetes - path: /v1.1/docs/design/security + path: /v1.1/docs/design/security/ - title: Kubernetes Identity and Access Management - path: /v1.1/docs/design/access + path: /v1.1/docs/design/access/ - title: Security Contexts - path: /v1.1/docs/design/security_context + path: /v1.1/docs/design/security_context/ - title: Kubernetes OpenVSwitch GRE/VxLAN networking - path: /v1.1/docs/admin/ovs-networking \ No newline at end of file + path: /v1.1/docs/admin/ovs-networking/ \ No newline at end of file diff --git a/_data/v1_1/samples.yml b/_data/v1_1/samples.yml index 7b616de605..a580ddf7fd 100644 --- a/_data/v1_1/samples.yml +++ b/_data/v1_1/samples.yml @@ -1,50 +1,52 @@ bigheader: "Samples" abstract: "A collection of example applications that show how to use Kubernetes." toc: +- title: Samples Overview + path: /v1.1/samples/ + - title: Clustered Application Samples - path: /v1.1/samples section: - title: Apache Cassandra Database - path: /v1.1/examples/cassandra/README + path: /v1.1/examples/cassandra/README/ - title: Apache Spark - path: /v1.1/examples/spark/README + path: /v1.1/examples/spark/README/ - title: Apache Storm - path: /v1.1/examples/storm/README + path: /v1.1/examples/storm/README/ - title: Distributed Task Queue - path: /v1.1/examples/celery-rabbitmq/README + path: /v1.1/examples/celery-rabbitmq/README/ - title: Hazelcast - path: /v1.1/examples/hazelcast/README + path: /v1.1/examples/hazelcast/README/ - title: Meteor Applications - path: /v1.1/examples/meteor/README + path: /v1.1/examples/meteor/README/ - title: Redis - path: /v1.1/examples/redis/README + path: /v1.1/examples/redis/README/ - title: RethinkDB - path: /v1.1/examples/rethinkdb/README + path: /v1.1/examples/rethinkdb/README/ - title: Elasticsearch/Kibana Logging Demonstration - path: /v1.1/docs/user-guide/logging-demo/README + path: /v1.1/docs/user-guide/logging-demo/README/ - title: Elasticsearch - path: /v1.1/examples/elasticsearch/README + path: /v1.1/examples/elasticsearch/README/ - title: OpenShift Origin - path: /v1.1/examples/openshift-origin/README + path: /v1.1/examples/openshift-origin/README/ - title: Ceph - path: /v1.1/examples/rbd/README + path: /v1.1/examples/rbd/README/ - title: Persistent Volume Samples section: - title: WordPress on a Kubernetes Persistent Volume - path: /v1.1/examples/mysql-wordpress-pd/README + path: /v1.1/examples/mysql-wordpress-pd/README/ - title: GlusterFS - path: /v1.1/examples/glusterfs/README + path: /v1.1/examples/glusterfs/README/ - title: iSCSI - path: /v1.1/examples/iscsi/README + path: /v1.1/examples/iscsi/README/ - title: NFS - path: /v1.1/examples/nfs/README + path: /v1.1/examples/nfs/README/ - title: Multi-tier Application Samples section: - title: Guestbook - Go Server - path: /v1.1/examples/guestbook-go/README + path: /v1.1/examples/guestbook-go/README/ - title: GuestBook - PHP Server - path: /v1.1/examples/guestbook/README + path: /v1.1/examples/guestbook/README/ - title: MySQL - Phabricator Server - path: /v1.1/examples/phabricator/README \ No newline at end of file + path: /v1.1/examples/phabricator/README/ \ No newline at end of file diff --git a/_data/v1_1/support.yml b/_data/v1_1/support.yml index f6a62c59dc..65d67dcc09 100644 --- a/_data/v1_1/support.yml +++ b/_data/v1_1/support.yml @@ -1,46 +1,48 @@ bigheader: "Support" abstract: "Troubleshooting resources, frequently asked questions, and community support channels." toc: +- title: Support Overview + path: /v1.1/support/ + - title: Troubleshooting - path: /v1.1/support section: - title: Web Interface - path: /v1.1/docs/user-guide/ui + path: /v1.1/docs/user-guide/ui/ - title: Logging - path: /v1.1/docs/user-guide/logging + path: /v1.1/docs/user-guide/logging/ - title: Container Access (exec) - path: /v1.1/docs/user-guide/getting-into-containers + path: /v1.1/docs/user-guide/getting-into-containers/ - title: Connect with Proxies - path: /v1.1/docs/user-guide/connecting-to-applications-proxy + path: /v1.1/docs/user-guide/connecting-to-applications-proxy/ - title: Connect with Port Forwarding - path: /v1.1/docs/user-guide/connecting-to-applications-port-forward + path: /v1.1/docs/user-guide/connecting-to-applications-port-forward/ - title: Troubleshooting Applications - path: /v1.1/docs/user-guide/application-troubleshooting + path: /v1.1/docs/user-guide/application-troubleshooting/ - title: Troubleshooting Clusters - path: /v1.1/docs/admin/cluster-troubleshooting + path: /v1.1/docs/admin/cluster-troubleshooting/ - title: Best Practices for Configuration - path: /v1.1/docs/user-guide/config-best-practices + path: /v1.1/docs/user-guide/config-best-practices/ - title: Frequently Asked Questions section: - title: User FAQ - path: https://github.com/kubernetes/kubernetes/wiki/User-FAQ + path: https://github.com/kubernetes/kubernetes/wiki/User-FAQ/ - title: Debugging FAQ - path: https://github.com/kubernetes/kubernetes/wiki/Debugging-FAQ + path: https://github.com/kubernetes/kubernetes/wiki/Debugging-FAQ/ - title: Services FAQ - path: https://github.com/kubernetes/kubernetes/wiki/Services-FAQ + path: https://github.com/kubernetes/kubernetes/wiki/Services-FAQ/ - title: Other Resources section: - title: Known Issues - path: /v1.1/docs/user-guide/known-issues + path: /v1.1/docs/user-guide/known-issues/ - title: Kubernetes Issue Tracker on GitHub - path: https://github.com/kubernetes/kubernetes/issues + path: https://github.com/kubernetes/kubernetes/issues/ - title: Report a Security Vulnerability - path: /v1.1/docs/reporting-security-issues + path: /v1.1/docs/reporting-security-issues/ - title: Release Notes - path: https://github.com/kubernetes/kubernetes/releases + path: https://github.com/kubernetes/kubernetes/releases/ - title: Release Roadmap - path: /v1.1/docs/roadmap + path: /v1.1/docs/roadmap/ - title: Contributing to Kubernetes Documentation - path: /v1.1/editdocs + path: /v1.1/editdocs/ diff --git a/_includes/pagetoc.html b/_includes/pagetoc.html index b6ff9446f5..416228cef5 100644 --- a/_includes/pagetoc.html +++ b/_includes/pagetoc.html @@ -1 +1,2 @@ +

Table of contents

\ No newline at end of file diff --git a/_includes/tocsearch.html b/_includes/tocsearch.html new file mode 100644 index 0000000000..f513451ce9 --- /dev/null +++ b/_includes/tocsearch.html @@ -0,0 +1 @@ +{% for item in tree %}{% if item.section %}{% assign tree = item.section %}{% include tocsearch.html %}{% else %}{% if item.path == page.url %}{% assign foundTOC = thistoc %}{% break %}{% endif %}{% endif %}{% endfor %} \ No newline at end of file diff --git a/_includes/tree.html b/_includes/tree.html index 7838cc6755..80fc1be3c1 100644 --- a/_includes/tree.html +++ b/_includes/tree.html @@ -3,4 +3,4 @@
{% assign tree = item.section %}{% include tree.html %}
{% else %} - {% endif %}{% endfor %} + {% endif %}{% endfor %} \ No newline at end of file diff --git a/_layouts/docwithnav.html b/_layouts/docwithnav.html index 8cede30dc0..f3d9013760 100755 --- a/_layouts/docwithnav.html +++ b/_layouts/docwithnav.html @@ -24,8 +24,8 @@ layout: headerfooter
- {% assign tree = site.data[page.versionfilesafe][page.section].toc %} - {% include tree.html %} + {% for thistoc in site.data[page.versionfilesafe].globals.tocs %}{% assign tree = site.data[page.versionfilesafe][thistoc].toc %}{% include tocsearch.html %}{% endfor %} + {% if foundTOC %}{% assign tree = site.data[page.versionfilesafe][foundTOC].toc %}{% include tree.html %}{% endif %}
{% if page.showedit == true %}Edit This Page{% endif %} diff --git a/v1.1/api-ref.md b/v1.1/api-ref.md index 399ad39de0..b493603842 100644 --- a/v1.1/api-ref.md +++ b/v1.1/api-ref.md @@ -2,11 +2,9 @@ title: "Kubernetes API Reference" --- -## {{ page.title }} ## - Use these reference documents to learn how to interact with Kubernetes through the REST API. -You can also view details about the *Extensions API*. For more about extensions, see [API versioning](docs/api.html). +You can also view details about the *Extensions API*. For more about extensions, see [API versioning](/{{page.version}}/docs/api).

Table of Contents:

    diff --git a/v1.1/app-admin-detailed.md b/v1.1/app-admin-detailed.md index ae0c46d3a8..9968af7946 100644 --- a/v1.1/app-admin-detailed.md +++ b/v1.1/app-admin-detailed.md @@ -2,8 +2,6 @@ title: "Application Administration: Detailed Walkthrough" --- -## {{ page.title }} ## - The detailed walkthrough covers all the in-depth details and tasks for administering your applications in Kubernetes.

    Table of Contents:

    diff --git a/v1.1/basicstutorials.md b/v1.1/basicstutorials.md index e81620bb5b..06af1b9c89 100644 --- a/v1.1/basicstutorials.md +++ b/v1.1/basicstutorials.md @@ -2,8 +2,6 @@ title: "Quick Walkthrough: Kubernetes Basics" --- -## {{ page.title }} ## - Use this quick walkthrough of Kubernetes to learn about the basic application administration tasks.

    Table of Contents:

    diff --git a/v1.1/deploy-clusters.md b/v1.1/deploy-clusters.md index 7f74683d24..7a3976d321 100644 --- a/v1.1/deploy-clusters.md +++ b/v1.1/deploy-clusters.md @@ -2,8 +2,6 @@ title: "Examples: Deploying Clusters" --- -## {{ page.title }} ## - Use the following examples to learn how to deploy your application into a Kubernetes cluster.

    Table of Contents:

    diff --git a/v1.1/docs/README.md b/v1.1/docs/README.md index 0f06e68e06..abed6e0046 100644 --- a/v1.1/docs/README.md +++ b/v1.1/docs/README.md @@ -1,21 +1,17 @@ --- title: "Kubernetes Documentation: releases.k8s.io/release-1.1" --- - - -# Kubernetes Documentation: releases.k8s.io/release-1.1 - -* The [User's guide](user-guide/README.html) is for anyone who wants to run programs and +* The [User's guide](user-guide/README) is for anyone who wants to run programs and services on an existing Kubernetes cluster. -* The [Cluster Admin's guide](admin/README.html) is for anyone setting up +* The [Cluster Admin's guide](admin/README) is for anyone setting up a Kubernetes cluster or administering it. -* The [Developer guide](devel/README.html) is for anyone wanting to write +* The [Developer guide](devel/README) is for anyone wanting to write programs that access the Kubernetes API, write plugins or extensions, or modify the core code of Kubernetes. -* The [Kubectl Command Line Interface](user-guide/kubectl/kubectl.html) is a detailed reference on +* The [Kubectl Command Line Interface](user-guide/kubectl/kubectl) is a detailed reference on the `kubectl` CLI. * The [API object documentation](http://kubernetes.io/third_party/swagger-ui/) @@ -26,10 +22,10 @@ title: "Kubernetes Documentation: releases.k8s.io/release-1.1" * There are example files and walkthroughs in the [examples](../examples/) folder. -* If something went wrong, see the [troubleshooting](troubleshooting.html) document for how to debug. -You should also check the [known issues](user-guide/known-issues.html) for the release you're using. +* If something went wrong, see the [troubleshooting](troubleshooting) document for how to debug. +You should also check the [known issues](user-guide/known-issues) for the release you're using. -* To report a security issue, see [Reporting a Security Issue](reporting-security-issues.html). +* To report a security issue, see [Reporting a Security Issue](reporting-security-issues). diff --git a/v1.1/docs/admin/README.md b/v1.1/docs/admin/README.md index f7e1803e01..6d672796c2 100644 --- a/v1.1/docs/admin/README.md +++ b/v1.1/docs/admin/README.md @@ -1,44 +1,40 @@ --- title: "Kubernetes Cluster Admin Guide" --- - - -# Kubernetes Cluster Admin Guide - The cluster admin guide is for anyone creating or administering a Kubernetes cluster. -It assumes some familiarity with concepts in the [User Guide](../user-guide/README.html). +It assumes some familiarity with concepts in the [User Guide](../user-guide/README). ## Admin Guide Table of Contents -[Introduction](introduction.html) +[Introduction](introduction) -1. [Components of a cluster](cluster-components.html) - 1. [Cluster Management](cluster-management.html) +1. [Components of a cluster](cluster-components) + 1. [Cluster Management](cluster-management) 1. Administrating Master Components - 1. [The kube-apiserver binary](kube-apiserver.html) - 1. [Authorization](authorization.html) - 1. [Authentication](authentication.html) - 1. [Accessing the api](accessing-the-api.html) - 1. [Admission Controllers](admission-controllers.html) - 1. [Administrating Service Accounts](service-accounts-admin.html) - 1. [Resource Quotas](resource-quota.html) - 1. [The kube-scheduler binary](kube-scheduler.html) - 1. [The kube-controller-manager binary](kube-controller-manager.html) - 1. [Administrating Kubernetes Nodes](node.html) - 1. [The kubelet binary](kubelet.html) - 1. [Garbage Collection](garbage-collection.html) - 1. [The kube-proxy binary](kube-proxy.html) + 1. [The kube-apiserver binary](kube-apiserver) + 1. [Authorization](authorization) + 1. [Authentication](authentication) + 1. [Accessing the api](accessing-the-api) + 1. [Admission Controllers](admission-controllers) + 1. [Administrating Service Accounts](service-accounts-admin) + 1. [Resource Quotas](resource-quota) + 1. [The kube-scheduler binary](kube-scheduler) + 1. [The kube-controller-manager binary](kube-controller-manager) + 1. [Administrating Kubernetes Nodes](node) + 1. [The kubelet binary](kubelet) + 1. [Garbage Collection](garbage-collection) + 1. [The kube-proxy binary](kube-proxy) 1. Administrating Addons - 1. [DNS](dns.html) - 1. [Networking](networking.html) - 1. [OVS Networking](ovs-networking.html) + 1. [DNS](dns) + 1. [Networking](networking) + 1. [OVS Networking](ovs-networking) 1. Example Configurations - 1. [Multiple Clusters](multi-cluster.html) - 1. [High Availability Clusters](high-availability.html) - 1. [Large Clusters](cluster-large.html) - 1. [Getting started from scratch](../getting-started-guides/scratch.html) - 1. [Kubernetes's use of salt](salt.html) - 1. [Troubleshooting](cluster-troubleshooting.html) + 1. [Multiple Clusters](multi-cluster) + 1. [High Availability Clusters](high-availability) + 1. [Large Clusters](cluster-large) + 1. [Getting started from scratch](../getting-started-guides/scratch) + 1. [Kubernetes's use of salt](salt) + 1. [Troubleshooting](cluster-troubleshooting) diff --git a/v1.1/docs/admin/accessing-the-api.md b/v1.1/docs/admin/accessing-the-api.md index a611e04056..ffab5e3507 100644 --- a/v1.1/docs/admin/accessing-the-api.md +++ b/v1.1/docs/admin/accessing-the-api.md @@ -1,17 +1,13 @@ --- title: "Configuring APIserver ports" --- - - -# Configuring APIserver ports - This document describes what ports the Kubernetes apiserver may serve on and how to reach them. The audience is cluster administrators who want to customize their cluster or understand the details. Most questions about accessing the cluster are covered -in [Accessing the cluster](../user-guide/accessing-the-cluster.html). +in [Accessing the cluster](../user-guide/accessing-the-cluster). ## Ports and IPs Served On @@ -30,10 +26,10 @@ By default the Kubernetes APIserver serves HTTP on 2 ports: - default is port 6443, change with `--secure-port` flag. - default IP is first non-localhost network interface, change with `--bind-address` flag. - serves HTTPS. Set cert with `--tls-cert-file` and key with `--tls-private-key-file` flag. - - uses token-file or client-certificate based [authentication](authentication.html). - - uses policy-based [authorization](authorization.html). + - uses token-file or client-certificate based [authentication](authentication). + - uses policy-based [authorization](authorization). 3. Removed: ReadOnly Port - - For security reasons, this had to be removed. Use the [service account](../user-guide/service-accounts.html) feature instead. + - For security reasons, this had to be removed. Use the [service account](../user-guide/service-accounts) feature instead. ## Proxies and Firewall rules @@ -56,7 +52,7 @@ variety of uses cases: running on the `kubernetes-master` machine. The proxy can use cert-based authentication or token-based authentication. 2. Processes running in Containers on Kubernetes that need to read from - the apiserver. Currently, these can use a [service account](../user-guide/service-accounts.html). + the apiserver. Currently, these can use a [service account](../user-guide/service-accounts). 3. Scheduler and Controller-manager processes, which need to do read-write API operations. Currently, these have to run on the same host as the apiserver and use the Localhost Port. In the future, these will be diff --git a/v1.1/docs/admin/admission-controllers.md b/v1.1/docs/admin/admission-controllers.md index 7e38cc89ce..bef19d5f45 100644 --- a/v1.1/docs/admin/admission-controllers.md +++ b/v1.1/docs/admin/admission-controllers.md @@ -1,33 +1,7 @@ --- title: "Admission Controllers" --- - - -# Admission Controllers - -**Table of Contents** - - -- [Admission Controllers](#admission-controllers) - - [What are they?](#what-are-they) - - [Why do I need them?](#why-do-i-need-them) - - [How do I turn on an admission control plug-in?](#how-do-i-turn-on-an-admission-control-plug-in) - - [What does each plug-in do?](#what-does-each-plug-in-do) - - [AlwaysAdmit](#alwaysadmit) - - [AlwaysDeny](#alwaysdeny) - - [DenyExecOnPrivileged (deprecated)](#denyexeconprivileged-deprecated) - - [DenyEscalatingExec](#denyescalatingexec) - - [ServiceAccount](#serviceaccount) - - [SecurityContextDeny](#securitycontextdeny) - - [ResourceQuota](#resourcequota) - - [LimitRanger](#limitranger) - - [InitialResources (experimental)](#initialresources-experimental) - - [NamespaceExists (deprecated)](#namespaceexists-deprecated) - - [NamespaceAutoProvision (deprecated)](#namespaceautoprovision-deprecated) - - [NamespaceLifecycle](#namespacelifecycle) - - [Is there a recommended set of plug-ins to use?](#is-there-a-recommended-set-of-plug-ins-to-use) - - +{% include pagetoc.html %} ## What are they? @@ -87,12 +61,12 @@ enabling this plug-in. ### ServiceAccount -This plug-in implements automation for [serviceAccounts](../user-guide/service-accounts.html). +This plug-in implements automation for [serviceAccounts](../user-guide/service-accounts). We strongly recommend using this plug-in if you intend to make use of Kubernetes `ServiceAccount` objects. ### SecurityContextDeny -This plug-in will deny any pod with a [SecurityContext](../user-guide/security-context.html) that defines options that were not available on the `Container`. +This plug-in will deny any pod with a [SecurityContext](../user-guide/security-context) that defines options that were not available on the `Container`. ### ResourceQuota @@ -100,7 +74,7 @@ This plug-in will observe the incoming request and ensure that it does not viola enumerated in the `ResourceQuota` object in a `Namespace`. If you are using `ResourceQuota` objects in your Kubernetes deployment, you MUST use this plug-in to enforce quota constraints. -See the [resourceQuota design doc](../design/admission_control_resource_quota.html) and the [example of Resource Quota](resourcequota/) for more details. +See the [resourceQuota design doc](../design/admission_control_resource_quota) and the [example of Resource Quota](resourcequota/) for more details. It is strongly encouraged that this plug-in is configured last in the sequence of admission control plug-ins. This is so that quota is not prematurely incremented only for the request to be rejected later in admission control. @@ -113,7 +87,7 @@ your Kubernetes deployment, you MUST use this plug-in to enforce those constrain be used to apply default resource requests to Pods that don't specify any; currently, the default LimitRanger applies a 0.1 CPU requirement to all Pods in the `default` namespace. -See the [limitRange design doc](../design/admission_control_limit_range.html) and the [example of Limit Range](limitrange/) for more details. +See the [limitRange design doc](../design/admission_control_limit_range) and the [example of Limit Range](limitrange/) for more details. ### InitialResources (experimental) @@ -122,7 +96,7 @@ then the plug-in auto-populates a compute resource request based on historical u If there is not enough data to make a decision the Request is left unchanged. When the plug-in sets a compute resource request, it annotates the pod with information on what compute resources it auto-populated. -See the [InitialResouces proposal](../proposals/initial-resources.html) for more details. +See the [InitialResouces proposal](../proposals/initial-resources) for more details. ### NamespaceExists (deprecated) @@ -154,9 +128,9 @@ Yes. For Kubernetes 1.0, we strongly recommend running the following set of admission control plug-ins (order matters): ``` -{% raw %} + --admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota -{% endraw %} + ``` diff --git a/v1.1/docs/admin/authentication.md b/v1.1/docs/admin/authentication.md index ac32e2b244..bf64b491a6 100644 --- a/v1.1/docs/admin/authentication.md +++ b/v1.1/docs/admin/authentication.md @@ -1,10 +1,6 @@ --- title: "Authentication Plugins" --- - - -# Authentication Plugins - Kubernetes uses client certificates, tokens, or http basic auth to authenticate users for API calls. **Client certificate authentication** is enabled by passing the `--client-ca-file=SOMEFILE` diff --git a/v1.1/docs/admin/authorization.md b/v1.1/docs/admin/authorization.md index b7c9fe66ee..5bea29d852 100644 --- a/v1.1/docs/admin/authorization.md +++ b/v1.1/docs/admin/authorization.md @@ -1,13 +1,8 @@ --- title: "Authorization Plugins" --- - - -# Authorization Plugins - - In Kubernetes, authorization happens as a separate step from authentication. -See the [authentication documentation](authentication.html) for an +See the [authentication documentation](authentication) for an overview of authentication. Authorization applies to all HTTP accesses on the main (secure) apiserver port. @@ -94,25 +89,25 @@ To permit an action Policy with an unset namespace applies regardless of namespa A service account automatically generates a user. The user's name is generated according to the naming convention: ``` -{% raw %} + system:serviceaccount:: -{% endraw %} + ``` Creating a new namespace also causes a new service account to be created, of this form:* ``` -{% raw %} + system:serviceaccount::default -{% endraw %} + ``` For example, if you wanted to grant the default service account in the kube-system full privilege to the API, you would add this line to your policy file: {% highlight json %} -{% raw %} + {"user":"system:serviceaccount:kube-system:default"} -{% endraw %} + {% endhighlight %} The apiserver will need to be restarted to pickup the new policy lines. @@ -123,11 +118,11 @@ Other implementations can be developed fairly easily. The APIserver calls the Authorizer interface: {% highlight go %} -{% raw %} + type Authorizer interface { Authorize(a Attributes) error } -{% endraw %} + {% endhighlight %} to determine whether or not to allow each API action. diff --git a/v1.1/docs/admin/cluster-components.md b/v1.1/docs/admin/cluster-components.md index 0be6bbb62c..389e4e19b2 100644 --- a/v1.1/docs/admin/cluster-components.md +++ b/v1.1/docs/admin/cluster-components.md @@ -1,10 +1,6 @@ --- title: "Kubernetes Cluster Admin Guide: Cluster Components" --- - - -# Kubernetes Cluster Admin Guide: Cluster Components - This document outlines the various binary components that need to run to deliver a functioning Kubernetes cluster. @@ -19,7 +15,7 @@ unsatisfied). Master components could in theory be run on any node in the cluster. However, for simplicity, current set up scripts typically start all master components on the same VM, and does not run user containers on this VM. See -[high-availability.md](high-availability.html) for an example multi-master-VM setup. +[high-availability.md](high-availability) for an example multi-master-VM setup. Even in the future, when Kubernetes is fully self-hosting, it will probably be wise to only allow master components to schedule on a subset of nodes, to limit @@ -28,19 +24,19 @@ node-compromising security exploit. ### kube-apiserver -[kube-apiserver](kube-apiserver.html) exposes the Kubernetes API; it is the front-end for the +[kube-apiserver](kube-apiserver) exposes the Kubernetes API; it is the front-end for the Kubernetes control plane. It is designed to scale horizontally (i.e., one scales -it by running more of them-- [high-availability.md](high-availability.html)). +it by running more of them-- [high-availability.md](high-availability)). ### etcd -[etcd](etcd.html) is used as Kubernetes' backing store. All cluster data is stored here. +[etcd](etcd) is used as Kubernetes' backing store. All cluster data is stored here. Proper administration of a Kubernetes cluster includes a backup plan for etcd's data. ### kube-controller-manager -[kube-controller-manager](kube-controller-manager.html) is a binary that runs controllers, which are the +[kube-controller-manager](kube-controller-manager) is a binary that runs controllers, which are the background threads that handle routine tasks in the cluster. Logically, each controller is a separate process, but to reduce the number of moving pieces in the system, they are all compiled into a single binary and run in a single @@ -61,7 +57,7 @@ These controllers include: ### kube-scheduler -[kube-scheduler](kube-scheduler.html) watches newly created pods that have no node assigned, and +[kube-scheduler](kube-scheduler) watches newly created pods that have no node assigned, and selects a node for them to run on. ### addons @@ -89,7 +85,7 @@ the Kubernetes runtime environment. ### kubelet -[kubelet](kubelet.html) is the primary node agent. It: +[kubelet](kubelet) is the primary node agent. It: * Watches for pods that have been assigned to its node (either by apiserver or via local configuration file) and: * Mounts the pod's required volumes @@ -102,7 +98,7 @@ the Kubernetes runtime environment. ### kube-proxy -[kube-proxy](kube-proxy.html) enables the Kubernetes service abstraction by maintaining +[kube-proxy](kube-proxy) enables the Kubernetes service abstraction by maintaining network rules on the host and performing connection forwarding. ### docker diff --git a/v1.1/docs/admin/cluster-large.md b/v1.1/docs/admin/cluster-large.md index 032a1dac5c..79beb98fae 100644 --- a/v1.1/docs/admin/cluster-large.md +++ b/v1.1/docs/admin/cluster-large.md @@ -1,71 +1,68 @@ ---- -title: "Using Large Clusters" -section: guides ---- - -## Support - -At v1.0, Kubernetes supports clusters up to 100 nodes with 30 pods per node and 1-2 containers per pod. - -## Table of Contents - -{% include pagetoc.html %} - -## Setup - -A cluster is a set of nodes (physical or virtual machines) running Kubernetes agents, managed by a "master" (the cluster-level control plane). - -Normally the number of nodes in a cluster is controlled by the the value `NUM_MINIONS` in the platform-specific `config-default.sh` file (for example, see [GCE's `config-default.sh`](http://releases.k8s.io/release-1.1/cluster/gce/config-default.sh)). - -Simply changing that value to something very large, however, may cause the setup script to fail for many cloud providers. A GCE deployment, for example, will run in to quota issues and fail to bring the cluster up. - -When setting up a large Kubernetes cluster, the following issues must be considered. - -### Quota Issues - -To avoid running into cloud provider quota issues, when creating a cluster with many nodes, consider: -* Increase the quota for things like CPU, IPs, etc. - * In [GCE, for example,](https://cloud.google.com/compute/docs/resource-quotas) you'll want to increase the quota for: - * CPUs - * VM instances - * Total persistent disk reserved - * In-use IP addresses - * Firewall Rules - * Forwarding rules - * Routes - * Target pools -* Gating the setup script so that it brings up new node VMs in smaller batches with waits in between, because some cloud providers rate limit the creation of VMs. - -### Addon Resources - -To prevent memory leaks or other resource issues in [cluster addons](https://releases.k8s.io/release-1.1/cluster/addons) from consuming all the resources available on a node, Kubernetes sets resource limits on addon containers to limit the CPU and Memory resources they can consume (See PR [#10653](http://pr.k8s.io/10653/files) and [#10778](http://pr.k8s.io/10778/files)). - -For example: - -{% highlight yaml %} -{% raw %} -containers: - - image: gcr.io/google_containers/heapster:v0.15.0 - name: heapster - resources: - limits: - cpu: 100m - memory: 200Mi -{% endraw %} -{% endhighlight %} - -These limits, however, are based on data collected from addons running on 4-node clusters (see [#10335](http://issue.k8s.io/10335#issuecomment-117861225)). The addons consume a lot more resources when running on large deployment clusters (see [#5880](http://issue.k8s.io/5880#issuecomment-113984085)). So, if a large cluster is deployed without adjusting these values, the addons may continuously get killed because they keep hitting the limits. - -To avoid running into cluster addon resource issues, when creating a cluster with many nodes, consider the following: -* Scale memory and CPU limits for each of the following addons, if used, along with the size of cluster (there is one replica of each handling the entire cluster so memory and CPU usage tends to grow proportionally with size/load on cluster): - * Heapster ([GCM/GCL backed](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/google/heapster-controller.yaml), [InfluxDB backed](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/influxdb/heapster-controller.yaml), [InfluxDB/GCL backed](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/googleinfluxdb/heapster-controller-combined.yaml), [standalone](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/standalone/heapster-controller.yaml)) - * [InfluxDB and Grafana](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/influxdb/influxdb-grafana-controller.yaml) - * [skydns, kube2sky, and dns etcd](http://releases.k8s.io/release-1.1/cluster/addons/dns/skydns-rc.yaml.in) - * [Kibana](http://releases.k8s.io/release-1.1/cluster/addons/fluentd-elasticsearch/kibana-controller.yaml) -* Scale number of replicas for the following addons, if used, along with the size of cluster (there are multiple replicas of each so increasing replicas should help handle increased load, but, since load per replica also increases slightly, also consider increasing CPU/memory limits): - * [elasticsearch](http://releases.k8s.io/release-1.1/cluster/addons/fluentd-elasticsearch/es-controller.yaml) -* Increase memory and CPU limits slightly for each of the following addons, if used, along with the size of cluster (there is one replica per node but CPU/memory usage increases slightly along with cluster load/size as well): - * [FluentD with ElasticSearch Plugin](http://releases.k8s.io/release-1.1/cluster/saltbase/salt/fluentd-es/fluentd-es.yaml) - * [FluentD with GCP Plugin](http://releases.k8s.io/release-1.1/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml) - +--- +title: "Using Large Clusters" +--- + +## Support + +At v1.0, Kubernetes supports clusters up to 100 nodes with 30 pods per node and 1-2 containers per pod. + + + +{% include pagetoc.html %} + +## Setup + +A cluster is a set of nodes (physical or virtual machines) running Kubernetes agents, managed by a "master" (the cluster-level control plane). + +Normally the number of nodes in a cluster is controlled by the the value `NUM_MINIONS` in the platform-specific `config-default.sh` file (for example, see [GCE's `config-default.sh`](http://releases.k8s.io/release-1.1/cluster/gce/config-default.sh)). + +Simply changing that value to something very large, however, may cause the setup script to fail for many cloud providers. A GCE deployment, for example, will run in to quota issues and fail to bring the cluster up. + +When setting up a large Kubernetes cluster, the following issues must be considered. + +### Quota Issues + +To avoid running into cloud provider quota issues, when creating a cluster with many nodes, consider: +* Increase the quota for things like CPU, IPs, etc. + * In [GCE, for example,](https://cloud.google.com/compute/docs/resource-quotas) you'll want to increase the quota for: + * CPUs + * VM instances + * Total persistent disk reserved + * In-use IP addresses + * Firewall Rules + * Forwarding rules + * Routes + * Target pools +* Gating the setup script so that it brings up new node VMs in smaller batches with waits in between, because some cloud providers rate limit the creation of VMs. + +### Addon Resources + +To prevent memory leaks or other resource issues in [cluster addons](https://releases.k8s.io/release-1.1/cluster/addons) from consuming all the resources available on a node, Kubernetes sets resource limits on addon containers to limit the CPU and Memory resources they can consume (See PR [#10653](http://pr.k8s.io/10653/files) and [#10778](http://pr.k8s.io/10778/files)). + +For example: + +{% highlight yaml %} +containers: + - image: gcr.io/google_containers/heapster:v0.15.0 + name: heapster + resources: + limits: + cpu: 100m + memory: 200Mi +{% endhighlight %} + +These limits, however, are based on data collected from addons running on 4-node clusters (see [#10335](http://issue.k8s.io/10335#issuecomment-117861225)). The addons consume a lot more resources when running on large deployment clusters (see [#5880](http://issue.k8s.io/5880#issuecomment-113984085)). So, if a large cluster is deployed without adjusting these values, the addons may continuously get killed because they keep hitting the limits. + +To avoid running into cluster addon resource issues, when creating a cluster with many nodes, consider the following: +* Scale memory and CPU limits for each of the following addons, if used, along with the size of cluster (there is one replica of each handling the entire cluster so memory and CPU usage tends to grow proportionally with size/load on cluster): + * Heapster ([GCM/GCL backed](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/google/heapster-controller.yaml), [InfluxDB backed](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/influxdb/heapster-controller.yaml), [InfluxDB/GCL backed](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/googleinfluxdb/heapster-controller-combined.yaml), [standalone](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/standalone/heapster-controller.yaml)) + * [InfluxDB and Grafana](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/influxdb/influxdb-grafana-controller.yaml) + * [skydns, kube2sky, and dns etcd](http://releases.k8s.io/release-1.1/cluster/addons/dns/skydns-rc.yaml.in) + * [Kibana](http://releases.k8s.io/release-1.1/cluster/addons/fluentd-elasticsearch/kibana-controller.yaml) +* Scale number of replicas for the following addons, if used, along with the size of cluster (there are multiple replicas of each so increasing replicas should help handle increased load, but, since load per replica also increases slightly, also consider increasing CPU/memory limits): + * [elasticsearch](http://releases.k8s.io/release-1.1/cluster/addons/fluentd-elasticsearch/es-controller.yaml) +* Increase memory and CPU limits slightly for each of the following addons, if used, along with the size of cluster (there is one replica per node but CPU/memory usage increases slightly along with cluster load/size as well): + * [FluentD with ElasticSearch Plugin](http://releases.k8s.io/release-1.1/cluster/saltbase/salt/fluentd-es/fluentd-es.yaml) + * [FluentD with GCP Plugin](http://releases.k8s.io/release-1.1/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml) + For directions on how to detect if addon containers are hitting resource limits, see the [Troubleshooting section of Compute Resources](../user-guide/compute-resources.html#troubleshooting). \ No newline at end of file diff --git a/v1.1/docs/admin/cluster-management.md b/v1.1/docs/admin/cluster-management.md index 9e2b756828..c86ca7f167 100644 --- a/v1.1/docs/admin/cluster-management.md +++ b/v1.1/docs/admin/cluster-management.md @@ -1,10 +1,6 @@ --- title: "Cluster Management" --- - - -# Cluster Management - This document describes several topics related to the lifecycle of a cluster: creating a new cluster, upgrading your cluster's master and worker nodes, performing node maintenance (e.g. kernel upgrades), and upgrading the Kubernetes API version of a @@ -12,7 +8,7 @@ running cluster. ## Creating and configuring a Cluster -To install Kubernetes on a set of machines, consult one of the existing [Getting Started guides](../../docs/getting-started-guides/README.html) depending on your environment. +To install Kubernetes on a set of machines, consult one of the existing [Getting Started guides](/{{page.version}}/docs/getting-started-guides/README) depending on your environment. ## Upgrading a cluster @@ -51,17 +47,17 @@ Get its usage by running `cluster/gce/upgrade.sh -h`. For example, to upgrade just your master to a specific version (v1.0.2): {% highlight console %} -{% raw %} + cluster/gce/upgrade.sh -M v1.0.2 -{% endraw %} + {% endhighlight %} Alternatively, to upgrade your entire cluster to the latest stable release: {% highlight console %} -{% raw %} + cluster/gce/upgrade.sh release/stable -{% endraw %} + {% endhighlight %} ### Other platforms @@ -75,9 +71,9 @@ If your cluster runs short on resources you can easily add more machines to it i If you're using GCE or GKE it's done by resizing Instance Group managing your Nodes. It can be accomplished by modifying number of instances on `Compute > Compute Engine > Instance groups > your group > Edit group` [Google Cloud Console page](https://console.developers.google.com) or using gcloud CLI: ``` -{% raw %} + gcloud compute instance-groups managed --zone compute-zone resize my-cluster-minon-group --new-size 42 -{% endraw %} + ``` Instance Group will take care of putting appropriate image on new machines and start them, while Kubelet will register its Node with API server to make it available for scheduling. If you scale the instance group down, system will randomly choose Nodes to kill. @@ -105,9 +101,9 @@ The initial values of the autoscaler parameters set by ``kube-up.sh`` and some m or using gcloud CLI: ``` -{% raw %} + gcloud preview autoscaler --zone compute-zone -{% endraw %} + ``` Note that autoscaling will work properly only if node metrics are accessible in Google Cloud Monitoring. @@ -127,9 +123,9 @@ If you want more control over the upgrading process, you may use the following w Mark the node to be rebooted as unschedulable: {% highlight console %} -{% raw %} + kubectl replace nodes $NODENAME --patch='{"apiVersion": "v1", "spec": {"unschedulable": true}}' -{% endraw %} + {% endhighlight %} This keeps new pods from landing on the node while you are trying to get them off. @@ -139,9 +135,9 @@ Get the pods off the machine, via any of the following strategies: * Delete pods with: {% highlight console %} -{% raw %} + kubectl delete pods $PODNAME -{% endraw %} + {% endhighlight %} For pods with a replication controller, the pod will eventually be replaced by a new pod which will be scheduled to a new node. Additionally, if the pod is part of a service, then clients will automatically be redirected to the new pod. @@ -153,14 +149,14 @@ Perform maintenance work on the node. Make the node schedulable again: {% highlight console %} -{% raw %} + kubectl replace nodes $NODENAME --patch='{"apiVersion": "v1", "spec": {"unschedulable": false}}' -{% endraw %} + {% endhighlight %} If you deleted the node's VM instance and created a new one, then a new schedulable node resource will be created automatically when you create a new VM instance (if you're using a cloud provider that supports -node discovery; currently this is only Google Compute Engine, not including CoreOS on Google Compute Engine using kube-register). See [Node](node.html) for more details. +node discovery; currently this is only Google Compute Engine, not including CoreOS on Google Compute Engine using kube-register). See [Node](node) for more details. ## Advanced Topics @@ -197,10 +193,10 @@ for changes to this variable to take effect. You can use the `kube-version-change` utility to convert config files between different API versions. {% highlight console %} -{% raw %} + $ hack/build-go.sh cmd/kube-version-change $ _output/local/go/bin/kube-version-change -i myPod.v1beta3.yaml -o myPod.v1.yaml -{% endraw %} + {% endhighlight %} diff --git a/v1.1/docs/admin/cluster-troubleshooting.md b/v1.1/docs/admin/cluster-troubleshooting.md index 1dae8c9e82..41fbb9c4eb 100644 --- a/v1.1/docs/admin/cluster-troubleshooting.md +++ b/v1.1/docs/admin/cluster-troubleshooting.md @@ -1,14 +1,10 @@ --- title: "Cluster Troubleshooting" --- - - -# Cluster Troubleshooting - This doc is about cluster troubleshooting; we assume you have already ruled out your application as the root cause of the problem you are experiencing. See -the [application troubleshooting guide](../user-guide/application-troubleshooting.html) for tips on application debugging. -You may also visit [troubleshooting document](../troubleshooting.html) for more information. +the [application troubleshooting guide](../user-guide/application-troubleshooting) for tips on application debugging. +You may also visit [troubleshooting document](../troubleshooting) for more information. ## Listing your cluster @@ -17,9 +13,9 @@ The first thing to debug in your cluster is if your nodes are all registered cor Run {% highlight sh %} -{% raw %} + kubectl get nodes -{% endraw %} + {% endhighlight %} And verify that all of the nodes you expect to see are present and that they are all in the `Ready` state. @@ -92,7 +88,7 @@ Mitigations: - Action use IaaS providers reliable storage (e.g GCE PD or AWS EBS volume) for VMs with apiserver+etcd - Mitigates: Apiserver backing storage lost -- Action: Use (experimental) [high-availability](high-availability.html) configuration +- Action: Use (experimental) [high-availability](high-availability) configuration - Mitigates: Master VM shutdown or master components (scheduler, API server, controller-managing) crashing - Will tolerate one or more simultaneous node or component failures - Mitigates: Apiserver backing storage (i.e., etcd's data directory) lost @@ -111,7 +107,7 @@ Mitigations: - Mitigates: Node shutdown - Mitigates: Kubelet software fault -- Action: [Multiple independent clusters](multi-cluster.html) (and avoid making risky changes to all clusters at once) +- Action: [Multiple independent clusters](multi-cluster) (and avoid making risky changes to all clusters at once) - Mitigates: Everything listed above. diff --git a/v1.1/docs/admin/daemons.md b/v1.1/docs/admin/daemons.md index 396c5d9629..ef9ac23f52 100644 --- a/v1.1/docs/admin/daemons.md +++ b/v1.1/docs/admin/daemons.md @@ -1,31 +1,7 @@ --- title: "Daemon Sets" --- - - -# Daemon Sets - -**Table of Contents** - - -- [Daemon Sets](#daemon-sets) - - [What is a _Daemon Set_?](#what-is-a-daemon-set) - - [Writing a DaemonSet Spec](#writing-a-daemonset-spec) - - [Required Fields](#required-fields) - - [Pod Template](#pod-template) - - [Pod Selector](#pod-selector) - - [Running Pods on Only Some Nodes](#running-pods-on-only-some-nodes) - - [How Daemon Pods are Scheduled](#how-daemon-pods-are-scheduled) - - [Communicating with DaemonSet Pods](#communicating-with-daemonset-pods) - - [Updating a DaemonSet](#updating-a-daemonset) - - [Alternatives to Daemon Set](#alternatives-to-daemon-set) - - [Init Scripts](#init-scripts) - - [Bare Pods](#bare-pods) - - [Static Pods](#static-pods) - - [Replication Controller](#replication-controller) - - [Caveats](#caveats) - - +{% include pagetoc.html %} ## What is a _Daemon Set_? @@ -49,8 +25,8 @@ but with different flags and/or different memory and cpu requests for different ### Required Fields As with all other Kubernetes config, a DaemonSet needs `apiVersion`, `kind`, and `metadata` fields. For -general information about working with config files, see [here](../user-guide/simple-yaml.html), -[here](../user-guide/configuring-containers.html), and [here](../user-guide/working-with-resources.html). +general information about working with config files, see [here](../user-guide/simple-yaml), +[here](../user-guide/configuring-containers), and [here](../user-guide/working-with-resources). A DaemonSet also needs a [`.spec`](../devel/api-conventions.html#spec-and-status) section. @@ -59,20 +35,20 @@ A DaemonSet also needs a [`.spec`](../devel/api-conventions.html#spec-and-status The `.spec.template` is the only required field of the `.spec`. The `.spec.template` is a [pod template](../user-guide/replication-controller.html#pod-template). -It has exactly the same schema as a [pod](../user-guide/pods.html), except +It has exactly the same schema as a [pod](../user-guide/pods), except it is nested and does not have an `apiVersion` or `kind`. In addition to required fields for a pod, a pod template in a DaemonSet has to specify appropriate labels (see [pod selector](#pod-selector)). -A pod template in a DaemonSet must have a [`RestartPolicy`](../user-guide/pod-states.html) +A pod template in a DaemonSet must have a [`RestartPolicy`](../user-guide/pod-states) equal to `Always`, or be unspecified, which defaults to `Always`. ### Pod Selector The `.spec.selector` field is a pod selector. It works the same as the `.spec.selector` of -a [ReplicationController](../user-guide/replication-controller.html) or -[Job](../user-guide/jobs.html). +a [ReplicationController](../user-guide/replication-controller) or +[Job](../user-guide/jobs). If the `.spec.selector` is specified, it must equal the `.spec.template.metadata.labels`. If not specified, the are default to be equal. Config with these unequal will be rejected by the API. @@ -87,7 +63,7 @@ a node for testing. If you specify a `.spec.template.spec.nodeSelector`, then the DaemonSet controller will create pods on nodes which match that [node -selector](../user-guide/node-selection/README.html). +selector](../user-guide/node-selection/README). If you do not specify a `.spec.template.spec.nodeSelector`, then the DaemonSet controller will create pods on all nodes. @@ -163,14 +139,14 @@ use a Daemon Set rather than creating individual pods. ### Static Pods It is possible to create pods by writing a file to a certain directory watched by Kubelet. These -are called [static pods](static-pods.html). +are called [static pods](static-pods). Unlike DaemonSet, static pods cannot be managed with kubectl or other Kubernetes API clients. Static pods do not depend on the apiserver, making them useful in cluster bootstrapping cases. Also, static pods may be deprecated in the future. ### Replication Controller -Daemon Set are similar to [Replication Controllers](../user-guide/replication-controller.html) in that +Daemon Set are similar to [Replication Controllers](../user-guide/replication-controller) in that they both create pods, and those pods have processes which are not expected to terminate (e.g. web servers, storage servers). diff --git a/v1.1/docs/admin/dns.md b/v1.1/docs/admin/dns.md index b6373956fa..7da8efb526 100644 --- a/v1.1/docs/admin/dns.md +++ b/v1.1/docs/admin/dns.md @@ -1,10 +1,6 @@ --- title: "DNS Integration with Kubernetes" --- - - -# DNS Integration with Kubernetes - As of Kubernetes 0.8, DNS is offered as a [cluster add-on](http://releases.k8s.io/release-1.1/cluster/addons/README.md). If enabled, a DNS Pod and Service will be scheduled on the cluster, and the kubelets will be configured to tell individual containers to use the DNS Service's IP to resolve DNS names. diff --git a/v1.1/docs/admin/etcd.md b/v1.1/docs/admin/etcd.md index c9d45f2b5e..d147f9d1cc 100644 --- a/v1.1/docs/admin/etcd.md +++ b/v1.1/docs/admin/etcd.md @@ -1,10 +1,6 @@ --- title: "etcd" --- - - -# etcd - [etcd](https://coreos.com/etcd/docs/2.0.12/) is a highly-available key value store which Kubernetes uses for persistent storage of all of its REST API objects. @@ -34,7 +30,7 @@ be run on master VMs. The default location that kubelet scans for manifests is ## Kubernetes's usage of etcd By default, Kubernetes objects are stored under the `/registry` key in etcd. -This path can be prefixed by using the [kube-apiserver](kube-apiserver.html) flag +This path can be prefixed by using the [kube-apiserver](kube-apiserver) flag `--etcd-prefix="/foo"`. `etcd` is the only place that Kubernetes keeps state. @@ -46,9 +42,9 @@ test key. On your master VM (or somewhere with firewalls configured such that you can talk to your cluster's etcd), try: {% highlight sh %} -{% raw %} + curl -fs -X PUT "http://${host}:${port}/v2/keys/_test" -{% endraw %} + {% endhighlight %} diff --git a/v1.1/docs/admin/garbage-collection.md b/v1.1/docs/admin/garbage-collection.md index 8830706f31..e7c37643d0 100644 --- a/v1.1/docs/admin/garbage-collection.md +++ b/v1.1/docs/admin/garbage-collection.md @@ -1,6 +1,5 @@ --- title: "Configuring Garbage Collection" -section: guides --- ### Introduction diff --git a/v1.1/docs/admin/high-availability.md b/v1.1/docs/admin/high-availability.md index fcb378250c..372e590ca2 100644 --- a/v1.1/docs/admin/high-availability.md +++ b/v1.1/docs/admin/high-availability.md @@ -1,241 +1,230 @@ ---- -title: "High Availability Kubernetes Clusters" -section: guides ---- - -## Introduction - -This document describes how to build a high-availability (HA) Kubernetes cluster. This is a fairly advanced topic. -Users who merely want to experiment with Kubernetes are encouraged to use configurations that are simpler to set up such as -the simple [Docker based single node cluster instructions](../../docs/getting-started-guides/docker.html), -or try [Google Container Engine](https://cloud.google.com/container-engine/) for hosted Kubernetes. - -Also, at this time high availability support for Kubernetes is not continuously tested in our end-to-end (e2e) testing. We will -be working to add this continuous testing, but for now the single-node master installations are more heavily tested. - -## Table of Contents - -{% include pagetoc.html %} - -## Overview - -Setting up a truly reliable, highly available distributed system requires a number of steps, it is akin to -wearing underwear, pants, a belt, suspenders, another pair of underwear, and another pair of pants. We go into each -of these steps in detail, but a summary is given here to help guide and orient the user. - -The steps involved are as follows: - * [Creating the reliable constituent nodes that collectively form our HA master implementation.](#reliable-nodes) - * [Setting up a redundant, reliable storage layer with clustered etcd.](#establishing-a-redundant-reliable-data-storage-layer) - * [Starting replicated, load balanced Kubernetes API servers](#replicated-api-servers) - * [Setting up master-elected Kubernetes scheduler and controller-manager daemons](#master-elected-components) - -Here's what the system should look like when it's finished: -![High availability Kubernetes diagram](high-availability/ha.png) - -Ready? Let's get started. - -## Initial set-up - -The remainder of this guide assumes that you are setting up a 3-node clustered master, where each machine is running some flavor of Linux. -Examples in the guide are given for Debian distributions, but they should be easily adaptable to other distributions. -Likewise, this set up should work whether you are running in a public or private cloud provider, or if you are running -on bare metal. - -The easiest way to implement an HA Kubernetes cluster is to start with an existing single-master cluster. The -instructions at [https://get.k8s.io](https://get.k8s.io) -describe easy installation for single-master clusters on a variety of platforms. - -## Reliable nodes - -On each master node, we are going to run a number of processes that implement the Kubernetes API. The first step in making these reliable is -to make sure that each automatically restarts when it fails. To achieve this, we need to install a process watcher. We choose to use -the `kubelet` that we run on each of the worker nodes. This is convenient, since we can use containers to distribute our binaries, we can -establish resource limits, and introspect the resource usage of each daemon. Of course, we also need something to monitor the kubelet -itself (insert who watches the watcher jokes here). For Debian systems, we choose monit, but there are a number of alternate -choices. For example, on systemd-based systems (e.g. RHEL, CentOS), you can run 'systemctl enable kubelet'. - -If you are extending from a standard Kubernetes installation, the `kubelet` binary should already be present on your system. You can run -`which kubelet` to determine if the binary is in fact installed. If it is not installed, -you should install the [kubelet binary](https://storage.googleapis.com/kubernetes-release/release/v0.19.3/bin/linux/amd64/kubelet), the -[kubelet init file](http://releases.k8s.io/release-1.1/cluster/saltbase/salt/kubelet/initd) and [high-availability/default-kubelet](high-availability/default-kubelet) -scripts. - -If you are using monit, you should also install the monit daemon (`apt-get install monit`) and the [high-availability/monit-kubelet](high-availability/monit-kubelet) and -[high-availability/monit-docker](high-availability/monit-docker) configs. - -On systemd systems you `systemctl enable kubelet` and `systemctl enable docker`. - - -## Establishing a redundant, reliable data storage layer - -The central foundation of a highly available solution is a redundant, reliable storage layer. The number one rule of high-availability is -to protect the data. Whatever else happens, whatever catches on fire, if you have the data, you can rebuild. If you lose the data, you're -done. - -Clustered etcd already replicates your storage to all master instances in your cluster. This means that to lose data, all three nodes would need -to have their physical (or virtual) disks fail at the same time. The probability that this occurs is relatively low, so for many people -running a replicated etcd cluster is likely reliable enough. You can add additional reliability by increasing the -size of the cluster from three to five nodes. If that is still insufficient, you can add -[even more redundancy to your storage layer](#even-more-reliable-storage). - -### Clustering etcd - -The full details of clustering etcd are beyond the scope of this document, lots of details are given on the -[etcd clustering page](https://github.com/coreos/etcd/blob/master/Documentation/clustering.md). This example walks through -a simple cluster set up, using etcd's built in discovery to build our cluster. - -First, hit the etcd discovery service to create a new token: - -{% highlight sh %} -{% raw %} -curl https://discovery.etcd.io/new?size=3 -{% endraw %} -{% endhighlight %} - -On each node, copy the [etcd.yaml](high-availability/etcd.yaml) file into `/etc/kubernetes/manifests/etcd.yaml` - -The kubelet on each node actively monitors the contents of that directory, and it will create an instance of the `etcd` -server from the definition of the pod specified in `etcd.yaml`. - -Note that in `etcd.yaml` you should substitute the token URL you got above for `${DISCOVERY_TOKEN}` on all three machines, -and you should substitute a different name (e.g. `node-1`) for ${NODE_NAME} and the correct IP address -for `${NODE_IP}` on each machine. - - -#### Validating your cluster - -Once you copy this into all three nodes, you should have a clustered etcd set up. You can validate with - -{% highlight sh %} -{% raw %} -etcdctl member list -{% endraw %} -{% endhighlight %} - -and - -{% highlight sh %} -{% raw %} -etcdctl cluster-health -{% endraw %} -{% endhighlight %} - -You can also validate that this is working with `etcdctl set foo bar` on one node, and `etcd get foo` -on a different node. - -### Even more reliable storage - -Of course, if you are interested in increased data reliability, there are further options which makes the place where etcd -installs it's data even more reliable than regular disks (belts *and* suspenders, ftw!). - -If you use a cloud provider, then they usually provide this -for you, for example [Persistent Disk](https://cloud.google.com/compute/docs/disks/persistent-disks) on the Google Cloud Platform. These -are block-device persistent storage that can be mounted onto your virtual machine. Other cloud providers provide similar solutions. - -If you are running on physical machines, you can also use network attached redundant storage using an iSCSI or NFS interface. -Alternatively, you can run a clustered file system like Gluster or Ceph. Finally, you can also run a RAID array on each physical machine. - -Regardless of how you choose to implement it, if you chose to use one of these options, you should make sure that your storage is mounted -to each machine. If your storage is shared between the three masters in your cluster, you should create a different directory on the storage -for each node. Throughout these instructions, we assume that this storage is mounted to your machine in `/var/etcd/data` - - -## Replicated API Servers - -Once you have replicated etcd set up correctly, we will also install the apiserver using the kubelet. - -### Installing configuration files - -First you need to create the initial log file, so that Docker mounts a file instead of a directory: - -{% highlight sh %} -{% raw %} -touch /var/log/kube-apiserver.log -{% endraw %} -{% endhighlight %} - -Next, you need to create a `/srv/kubernetes/` directory on each node. This directory includes: - * basic_auth.csv - basic auth user and password - * ca.crt - Certificate Authority cert - * known_tokens.csv - tokens that entities (e.g. the kubelet) can use to talk to the apiserver - * kubecfg.crt - Client certificate, public key - * kubecfg.key - Client certificate, private key - * server.cert - Server certificate, public key - * server.key - Server certificate, private key - -The easiest way to create this directory, may be to copy it from the master node of a working cluster, or you can manually generate these files yourself. - -### Starting the API Server - -Once these files exist, copy the [kube-apiserver.yaml](high-availability/kube-apiserver.yaml) into `/etc/kubernetes/manifests/` on each master node. - -The kubelet monitors this directory, and will automatically create an instance of the `kube-apiserver` container using the pod definition specified -in the file. - -### Load balancing - -At this point, you should have 3 apiservers all working correctly. If you set up a network load balancer, you should -be able to access your cluster via that load balancer, and see traffic balancing between the apiserver instances. Setting -up a load balancer will depend on the specifics of your platform, for example instructions for the Google Cloud -Platform can be found [here](https://cloud.google.com/compute/docs/load-balancing/) - -Note, if you are using authentication, you may need to regenerate your certificate to include the IP address of the balancer, -in addition to the IP addresses of the individual nodes. - -For pods that you deploy into the cluster, the `kubernetes` service/dns name should provide a load balanced endpoint for the master automatically. - -For external users of the API (e.g. the `kubectl` command line interface, continuous build pipelines, or other clients) you will want to configure -them to talk to the external load balancer's IP address. - -## Master elected components - -So far we have set up state storage, and we have set up the API server, but we haven't run anything that actually modifies -cluster state, such as the controller manager and scheduler. To achieve this reliably, we only want to have one actor modifying state at a time, but we want replicated -instances of these actors, in case a machine dies. To achieve this, we are going to use a lease-lock in etcd to perform -master election. On each of the three apiserver nodes, we run a small utility application named `podmaster`. It's job is to implement a master -election protocol using etcd "compare and swap". If the apiserver node wins the election, it starts the master component it is managing (e.g. the scheduler), if it -loses the election, it ensures that any master components running on the node (e.g. the scheduler) are stopped. - -In the future, we expect to more tightly integrate this lease-locking into the scheduler and controller-manager binaries directly, as described in the [high availability design proposal](../proposals/high-availability.html) - -### Installing configuration files - -First, create empty log files on each node, so that Docker will mount the files not make new directories: - -{% highlight sh %} -{% raw %} -touch /var/log/kube-scheduler.log -touch /var/log/kube-controller-manager.log -{% endraw %} -{% endhighlight %} - -Next, set up the descriptions of the scheduler and controller manager pods on each node. -by copying [kube-scheduler.yaml](high-availability/kube-scheduler.yaml) and [kube-controller-manager.yaml](high-availability/kube-controller-manager.yaml) into the `/srv/kubernetes/` - directory. - -### Running the podmaster - -Now that the configuration files are in place, copy the [podmaster.yaml](high-availability/podmaster.yaml) config file into `/etc/kubernetes/manifests/` - -As before, the kubelet on the node monitors this directory, and will start an instance of the podmaster using the pod specification provided in `podmaster.yaml`. - -Now you will have one instance of the scheduler process running on a single master node, and likewise one -controller-manager process running on a single (possibly different) master node. If either of these processes fail, -the kubelet will restart them. If any of these nodes fail, the process will move to a different instance of a master -node. - -## Conclusion - -At this point, you are done (yeah!) with the master components, but you still need to add worker nodes (boo!). - -If you have an existing cluster, this is as simple as reconfiguring your kubelets to talk to the load-balanced endpoint, and -restarting the kubelets on each node. - -If you are turning up a fresh cluster, you will need to install the kubelet and kube-proxy on each worker node, and -set the `--apiserver` flag to your replicated endpoint. - -## Vagrant up! - -We indeed have an initial proof of concept tester for this, which is available [here](https://releases.k8s.io/release-1.1/examples/high-availability). - +--- +title: "High Availability Kubernetes Clusters" +--- + +## Introduction + +This document describes how to build a high-availability (HA) Kubernetes cluster. This is a fairly advanced topic. +Users who merely want to experiment with Kubernetes are encouraged to use configurations that are simpler to set up such as +the simple [Docker based single node cluster instructions](/{{page.version}}/docs/getting-started-guides/docker), +or try [Google Container Engine](https://cloud.google.com/container-engine/) for hosted Kubernetes. + +Also, at this time high availability support for Kubernetes is not continuously tested in our end-to-end (e2e) testing. We will +be working to add this continuous testing, but for now the single-node master installations are more heavily tested. + + + +{% include pagetoc.html %} + +## Overview + +Setting up a truly reliable, highly available distributed system requires a number of steps, it is akin to +wearing underwear, pants, a belt, suspenders, another pair of underwear, and another pair of pants. We go into each +of these steps in detail, but a summary is given here to help guide and orient the user. + +The steps involved are as follows: + * [Creating the reliable constituent nodes that collectively form our HA master implementation.](#reliable-nodes) + * [Setting up a redundant, reliable storage layer with clustered etcd.](#establishing-a-redundant-reliable-data-storage-layer) + * [Starting replicated, load balanced Kubernetes API servers](#replicated-api-servers) + * [Setting up master-elected Kubernetes scheduler and controller-manager daemons](#master-elected-components) + +Here's what the system should look like when it's finished: +![High availability Kubernetes diagram](high-availability/ha.png) + +Ready? Let's get started. + +## Initial set-up + +The remainder of this guide assumes that you are setting up a 3-node clustered master, where each machine is running some flavor of Linux. +Examples in the guide are given for Debian distributions, but they should be easily adaptable to other distributions. +Likewise, this set up should work whether you are running in a public or private cloud provider, or if you are running +on bare metal. + +The easiest way to implement an HA Kubernetes cluster is to start with an existing single-master cluster. The +instructions at [https://get.k8s.io](https://get.k8s.io) +describe easy installation for single-master clusters on a variety of platforms. + +## Reliable nodes + +On each master node, we are going to run a number of processes that implement the Kubernetes API. The first step in making these reliable is +to make sure that each automatically restarts when it fails. To achieve this, we need to install a process watcher. We choose to use +the `kubelet` that we run on each of the worker nodes. This is convenient, since we can use containers to distribute our binaries, we can +establish resource limits, and introspect the resource usage of each daemon. Of course, we also need something to monitor the kubelet +itself (insert who watches the watcher jokes here). For Debian systems, we choose monit, but there are a number of alternate +choices. For example, on systemd-based systems (e.g. RHEL, CentOS), you can run 'systemctl enable kubelet'. + +If you are extending from a standard Kubernetes installation, the `kubelet` binary should already be present on your system. You can run +`which kubelet` to determine if the binary is in fact installed. If it is not installed, +you should install the [kubelet binary](https://storage.googleapis.com/kubernetes-release/release/v0.19.3/bin/linux/amd64/kubelet), the +[kubelet init file](http://releases.k8s.io/release-1.1/cluster/saltbase/salt/kubelet/initd) and [high-availability/default-kubelet](high-availability/default-kubelet) +scripts. + +If you are using monit, you should also install the monit daemon (`apt-get install monit`) and the [high-availability/monit-kubelet](high-availability/monit-kubelet) and +[high-availability/monit-docker](high-availability/monit-docker) configs. + +On systemd systems you `systemctl enable kubelet` and `systemctl enable docker`. + + +## Establishing a redundant, reliable data storage layer + +The central foundation of a highly available solution is a redundant, reliable storage layer. The number one rule of high-availability is +to protect the data. Whatever else happens, whatever catches on fire, if you have the data, you can rebuild. If you lose the data, you're +done. + +Clustered etcd already replicates your storage to all master instances in your cluster. This means that to lose data, all three nodes would need +to have their physical (or virtual) disks fail at the same time. The probability that this occurs is relatively low, so for many people +running a replicated etcd cluster is likely reliable enough. You can add additional reliability by increasing the +size of the cluster from three to five nodes. If that is still insufficient, you can add +[even more redundancy to your storage layer](#even-more-reliable-storage). + +### Clustering etcd + +The full details of clustering etcd are beyond the scope of this document, lots of details are given on the +[etcd clustering page](https://github.com/coreos/etcd/blob/master/Documentation/clustering.md). This example walks through +a simple cluster set up, using etcd's built in discovery to build our cluster. + +First, hit the etcd discovery service to create a new token: + +{% highlight sh %} +curl https://discovery.etcd.io/new?size=3 +{% endhighlight %} + +On each node, copy the [etcd.yaml](high-availability/etcd.yaml) file into `/etc/kubernetes/manifests/etcd.yaml` + +The kubelet on each node actively monitors the contents of that directory, and it will create an instance of the `etcd` +server from the definition of the pod specified in `etcd.yaml`. + +Note that in `etcd.yaml` you should substitute the token URL you got above for `${DISCOVERY_TOKEN}` on all three machines, +and you should substitute a different name (e.g. `node-1`) for ${NODE_NAME} and the correct IP address +for `${NODE_IP}` on each machine. + + +#### Validating your cluster + +Once you copy this into all three nodes, you should have a clustered etcd set up. You can validate with + +{% highlight sh %} +etcdctl member list +{% endhighlight %} + +and + +{% highlight sh %} +etcdctl cluster-health +{% endhighlight %} + +You can also validate that this is working with `etcdctl set foo bar` on one node, and `etcd get foo` +on a different node. + +### Even more reliable storage + +Of course, if you are interested in increased data reliability, there are further options which makes the place where etcd +installs it's data even more reliable than regular disks (belts *and* suspenders, ftw!). + +If you use a cloud provider, then they usually provide this +for you, for example [Persistent Disk](https://cloud.google.com/compute/docs/disks/persistent-disks) on the Google Cloud Platform. These +are block-device persistent storage that can be mounted onto your virtual machine. Other cloud providers provide similar solutions. + +If you are running on physical machines, you can also use network attached redundant storage using an iSCSI or NFS interface. +Alternatively, you can run a clustered file system like Gluster or Ceph. Finally, you can also run a RAID array on each physical machine. + +Regardless of how you choose to implement it, if you chose to use one of these options, you should make sure that your storage is mounted +to each machine. If your storage is shared between the three masters in your cluster, you should create a different directory on the storage +for each node. Throughout these instructions, we assume that this storage is mounted to your machine in `/var/etcd/data` + + +## Replicated API Servers + +Once you have replicated etcd set up correctly, we will also install the apiserver using the kubelet. + +### Installing configuration files + +First you need to create the initial log file, so that Docker mounts a file instead of a directory: + +{% highlight sh %} +touch /var/log/kube-apiserver.log +{% endhighlight %} + +Next, you need to create a `/srv/kubernetes/` directory on each node. This directory includes: + * basic_auth.csv - basic auth user and password + * ca.crt - Certificate Authority cert + * known_tokens.csv - tokens that entities (e.g. the kubelet) can use to talk to the apiserver + * kubecfg.crt - Client certificate, public key + * kubecfg.key - Client certificate, private key + * server.cert - Server certificate, public key + * server.key - Server certificate, private key + +The easiest way to create this directory, may be to copy it from the master node of a working cluster, or you can manually generate these files yourself. + +### Starting the API Server + +Once these files exist, copy the [kube-apiserver.yaml](high-availability/kube-apiserver.yaml) into `/etc/kubernetes/manifests/` on each master node. + +The kubelet monitors this directory, and will automatically create an instance of the `kube-apiserver` container using the pod definition specified +in the file. + +### Load balancing + +At this point, you should have 3 apiservers all working correctly. If you set up a network load balancer, you should +be able to access your cluster via that load balancer, and see traffic balancing between the apiserver instances. Setting +up a load balancer will depend on the specifics of your platform, for example instructions for the Google Cloud +Platform can be found [here](https://cloud.google.com/compute/docs/load-balancing/) + +Note, if you are using authentication, you may need to regenerate your certificate to include the IP address of the balancer, +in addition to the IP addresses of the individual nodes. + +For pods that you deploy into the cluster, the `kubernetes` service/dns name should provide a load balanced endpoint for the master automatically. + +For external users of the API (e.g. the `kubectl` command line interface, continuous build pipelines, or other clients) you will want to configure +them to talk to the external load balancer's IP address. + +## Master elected components + +So far we have set up state storage, and we have set up the API server, but we haven't run anything that actually modifies +cluster state, such as the controller manager and scheduler. To achieve this reliably, we only want to have one actor modifying state at a time, but we want replicated +instances of these actors, in case a machine dies. To achieve this, we are going to use a lease-lock in etcd to perform +master election. On each of the three apiserver nodes, we run a small utility application named `podmaster`. It's job is to implement a master +election protocol using etcd "compare and swap". If the apiserver node wins the election, it starts the master component it is managing (e.g. the scheduler), if it +loses the election, it ensures that any master components running on the node (e.g. the scheduler) are stopped. + +In the future, we expect to more tightly integrate this lease-locking into the scheduler and controller-manager binaries directly, as described in the [high availability design proposal](../proposals/high-availability) + +### Installing configuration files + +First, create empty log files on each node, so that Docker will mount the files not make new directories: + +{% highlight sh %} +touch /var/log/kube-scheduler.log +touch /var/log/kube-controller-manager.log +{% endhighlight %} + +Next, set up the descriptions of the scheduler and controller manager pods on each node. +by copying [kube-scheduler.yaml](high-availability/kube-scheduler.yaml) and [kube-controller-manager.yaml](high-availability/kube-controller-manager.yaml) into the `/srv/kubernetes/` + directory. + +### Running the podmaster + +Now that the configuration files are in place, copy the [podmaster.yaml](high-availability/podmaster.yaml) config file into `/etc/kubernetes/manifests/` + +As before, the kubelet on the node monitors this directory, and will start an instance of the podmaster using the pod specification provided in `podmaster.yaml`. + +Now you will have one instance of the scheduler process running on a single master node, and likewise one +controller-manager process running on a single (possibly different) master node. If either of these processes fail, +the kubelet will restart them. If any of these nodes fail, the process will move to a different instance of a master +node. + +## Conclusion + +At this point, you are done (yeah!) with the master components, but you still need to add worker nodes (boo!). + +If you have an existing cluster, this is as simple as reconfiguring your kubelets to talk to the load-balanced endpoint, and +restarting the kubelets on each node. + +If you are turning up a fresh cluster, you will need to install the kubelet and kube-proxy on each worker node, and +set the `--apiserver` flag to your replicated endpoint. + +## Vagrant up! + +We indeed have an initial proof of concept tester for this, which is available [here](https://releases.k8s.io/release-1.1/examples/high-availability). + It implements the major concepts (with a few minor reductions for simplicity), of the podmaster HA implementation alongside a quick smoke test using k8petstore. \ No newline at end of file diff --git a/v1.1/docs/admin/index.md b/v1.1/docs/admin/index.md index f7e1803e01..6d672796c2 100644 --- a/v1.1/docs/admin/index.md +++ b/v1.1/docs/admin/index.md @@ -1,44 +1,40 @@ --- title: "Kubernetes Cluster Admin Guide" --- - - -# Kubernetes Cluster Admin Guide - The cluster admin guide is for anyone creating or administering a Kubernetes cluster. -It assumes some familiarity with concepts in the [User Guide](../user-guide/README.html). +It assumes some familiarity with concepts in the [User Guide](../user-guide/README). ## Admin Guide Table of Contents -[Introduction](introduction.html) +[Introduction](introduction) -1. [Components of a cluster](cluster-components.html) - 1. [Cluster Management](cluster-management.html) +1. [Components of a cluster](cluster-components) + 1. [Cluster Management](cluster-management) 1. Administrating Master Components - 1. [The kube-apiserver binary](kube-apiserver.html) - 1. [Authorization](authorization.html) - 1. [Authentication](authentication.html) - 1. [Accessing the api](accessing-the-api.html) - 1. [Admission Controllers](admission-controllers.html) - 1. [Administrating Service Accounts](service-accounts-admin.html) - 1. [Resource Quotas](resource-quota.html) - 1. [The kube-scheduler binary](kube-scheduler.html) - 1. [The kube-controller-manager binary](kube-controller-manager.html) - 1. [Administrating Kubernetes Nodes](node.html) - 1. [The kubelet binary](kubelet.html) - 1. [Garbage Collection](garbage-collection.html) - 1. [The kube-proxy binary](kube-proxy.html) + 1. [The kube-apiserver binary](kube-apiserver) + 1. [Authorization](authorization) + 1. [Authentication](authentication) + 1. [Accessing the api](accessing-the-api) + 1. [Admission Controllers](admission-controllers) + 1. [Administrating Service Accounts](service-accounts-admin) + 1. [Resource Quotas](resource-quota) + 1. [The kube-scheduler binary](kube-scheduler) + 1. [The kube-controller-manager binary](kube-controller-manager) + 1. [Administrating Kubernetes Nodes](node) + 1. [The kubelet binary](kubelet) + 1. [Garbage Collection](garbage-collection) + 1. [The kube-proxy binary](kube-proxy) 1. Administrating Addons - 1. [DNS](dns.html) - 1. [Networking](networking.html) - 1. [OVS Networking](ovs-networking.html) + 1. [DNS](dns) + 1. [Networking](networking) + 1. [OVS Networking](ovs-networking) 1. Example Configurations - 1. [Multiple Clusters](multi-cluster.html) - 1. [High Availability Clusters](high-availability.html) - 1. [Large Clusters](cluster-large.html) - 1. [Getting started from scratch](../getting-started-guides/scratch.html) - 1. [Kubernetes's use of salt](salt.html) - 1. [Troubleshooting](cluster-troubleshooting.html) + 1. [Multiple Clusters](multi-cluster) + 1. [High Availability Clusters](high-availability) + 1. [Large Clusters](cluster-large) + 1. [Getting started from scratch](../getting-started-guides/scratch) + 1. [Kubernetes's use of salt](salt) + 1. [Troubleshooting](cluster-troubleshooting) diff --git a/v1.1/docs/admin/introduction.md b/v1.1/docs/admin/introduction.md index 954ee6608b..e6c4a435bf 100644 --- a/v1.1/docs/admin/introduction.md +++ b/v1.1/docs/admin/introduction.md @@ -1,19 +1,18 @@ --- title: "Kubernetes Cluster Admin Guide" -section: guides --- The cluster admin guide is for anyone creating or administering a Kubernetes cluster. -It assumes some familiarity with concepts in the [User Guide](../user-guide/README.html). +It assumes some familiarity with concepts in the [User Guide](../user-guide/README). + -## Table of Contents {% include pagetoc.html %} ## Planning a cluster There are many different examples of how to setup a kubernetes cluster. Many of them are listed in this -[matrix](../getting-started-guides/README.html). We call each of the combinations in this matrix a *distro*. +[matrix](../getting-started-guides/README). We call each of the combinations in this matrix a *distro*. Before choosing a particular guide, here are some things to consider: @@ -30,52 +29,52 @@ Before choosing a particular guide, here are some things to consider: - Not all distros are maintained as actively. Prefer ones which are listed as tested on a more recent version of Kubernetes. - If you are configuring kubernetes on-premises, you will need to consider what [networking - model](networking.html) fits best. - - If you are designing for very high-availability, you may want [clusters in multiple zones](multi-cluster.html). + model](networking) fits best. + - If you are designing for very high-availability, you may want [clusters in multiple zones](multi-cluster). - You may want to familiarize yourself with the various - [components](cluster-components.html) needed to run a cluster. + [components](cluster-components) needed to run a cluster. ## Setting up a cluster -Pick one of the Getting Started Guides from the [matrix](../getting-started-guides/README.html) and follow it. +Pick one of the Getting Started Guides from the [matrix](../getting-started-guides/README) and follow it. If none of the Getting Started Guides fits, you may want to pull ideas from several of the guides. -One option for custom networking is *OpenVSwitch GRE/VxLAN networking* ([ovs-networking.md](ovs-networking.html)), which +One option for custom networking is *OpenVSwitch GRE/VxLAN networking* ([ovs-networking.md](ovs-networking)), which uses OpenVSwitch to set up networking between pods across Kubernetes nodes. If you are modifying an existing guide which uses Salt, this document explains [how Salt is used in the Kubernetes -project](salt.html). +project](salt). ## Managing a cluster, including upgrades -[Managing a cluster](cluster-management.html). +[Managing a cluster](cluster-management). ## Managing nodes -[Managing nodes](node.html). +[Managing nodes](node). ## Optional Cluster Services -* **DNS Integration with SkyDNS** ([dns.md](dns.html)): +* **DNS Integration with SkyDNS** ([dns.md](dns)): Resolving a DNS name directly to a Kubernetes service. -* **Logging** with [Kibana](../user-guide/logging.html) +* **Logging** with [Kibana](../user-guide/logging) ## Multi-tenant support -* **Resource Quota** ([resource-quota.md](resource-quota.html)) +* **Resource Quota** ([resource-quota.md](resource-quota)) ## Security -* **Kubernetes Container Environment** ([docs/user-guide/container-environment.md](../user-guide/container-environment.html)): +* **Kubernetes Container Environment** ([docs/user-guide/container-environment.md](../user-guide/container-environment)): Describes the environment for Kubelet managed containers on a Kubernetes node. -* **Securing access to the API Server** [accessing the api](accessing-the-api.html) +* **Securing access to the API Server** [accessing the api](accessing-the-api) -* **Authentication** [authentication](authentication.html) +* **Authentication** [authentication](authentication) -* **Authorization** [authorization](authorization.html) +* **Authorization** [authorization](authorization) -* **Admission Controllers** [admission_controllers](admission-controllers.html) \ No newline at end of file +* **Admission Controllers** [admission_controllers](admission-controllers) \ No newline at end of file diff --git a/v1.1/docs/admin/kube-apiserver.md b/v1.1/docs/admin/kube-apiserver.md deleted file mode 100644 index 10a5c2de36..0000000000 --- a/v1.1/docs/admin/kube-apiserver.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -title: "kube-apiserver" ---- - - -## kube-apiserver - - - -### Synopsis - - -The Kubernetes API server validates and configures data -for the api objects which include pods, services, replicationcontrollers, and -others. The API Server services REST operations and provides the frontend to the -cluster's shared state through which all other components interact. - -``` -{% raw %} -kube-apiserver -{% endraw %} -``` - -### Options - -``` -{% raw %} - --admission-control="AlwaysAdmit": Ordered list of plug-ins to do admission control of resources into cluster. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, DenyEscalatingExec, DenyExecOnPrivileged, InitialResources, LimitRanger, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, ResourceQuota, SecurityContextDeny, ServiceAccount - --admission-control-config-file="": File with admission control configuration. - --advertise-address=: The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used. - --allow-privileged[=false]: If true, allow privileged containers. - --authorization-mode="AlwaysAllow": Ordered list of plug-ins to do authorization on secure port. Comma-delimited list of: AlwaysAllow,AlwaysDeny,ABAC - --authorization-policy-file="": File with authorization policy in csv format, used with --authorization-mode=ABAC, on the secure port. - --basic-auth-file="": If set, the file that will be used to admit requests to the secure port of the API server via http basic authentication. - --bind-address=0.0.0.0: The IP address on which to serve the --read-only-port and --secure-port ports. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0). - --cert-dir="/var/run/kubernetes": The directory where the TLS certs are located (by default /var/run/kubernetes). If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. - --client-ca-file="": If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. - --cloud-config="": The path to the cloud provider configuration file. Empty string for no configuration file. - --cloud-provider="": The provider for cloud services. Empty string for no provider. - --cluster-name="kubernetes": The instance prefix for the cluster - --cors-allowed-origins=[]: List of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled. - --etcd-config="": The config file for the etcd client. Mutually exclusive with -etcd-servers. - --etcd-prefix="/registry": The prefix for all resource paths in etcd. - --etcd-servers=[]: List of etcd servers to watch (http://ip:port), comma separated. Mutually exclusive with -etcd-config - --etcd-servers-overrides=[]: Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are http://ip:port, semicolon separated. - --event-ttl=1h0m0s: Amount of time to retain events. Default 1 hour. - --experimental-keystone-url="": If passed, activates the keystone authentication plugin - --external-hostname="": The hostname to use when generating externalized URLs for this master (e.g. Swagger API Docs.) - --google-json-key="": The Google Cloud Platform Service Account JSON Key to use for authentication. - --insecure-bind-address=127.0.0.1: The IP address on which to serve the --insecure-port (set to 0.0.0.0 for all interfaces). Defaults to localhost. - --insecure-port=8080: The port on which to serve unsecured, unauthenticated access. Default 8080. It is assumed that firewall rules are set up such that this port is not reachable from outside of the cluster and that port 443 on the cluster's public address is proxied to this port. This is performed by nginx in the default setup. - --kubelet-certificate-authority="": Path to a cert. file for the certificate authority. - --kubelet-client-certificate="": Path to a client cert file for TLS. - --kubelet-client-key="": Path to a client key file for TLS. - --kubelet-https[=true]: Use https for kubelet connections - --kubelet-port=10250: Kubelet port - --kubelet-timeout=5s: Timeout for kubelet operations - --log-flush-frequency=5s: Maximum number of seconds between log flushes - --long-running-request-regexp="(/|^)((watch|proxy)(/|$)|(logs?|portforward|exec|attach)/?$)": A regular expression matching long running requests which should be excluded from maximum inflight request handling. - --master-service-namespace="default": The namespace from which the kubernetes master services should be injected into pods - --max-connection-bytes-per-sec=0: If non-zero, throttle each user connection to this number of bytes/sec. Currently only applies to long-running requests - --max-requests-inflight=400: The maximum number of requests in flight at a given time. When the server exceeds this, it rejects requests. Zero for no limit. - --min-request-timeout=1800: An optional field indicating the minimum number of seconds a handler must keep a request open before timing it out. Currently only honored by the watch request handler, which picks a randomized value above this number as the connection timeout, to spread out load. - --oidc-ca-file="": If set, the OpenID server's certificate will be verified by one of the authorities in the oidc-ca-file, otherwise the host's root CA set will be used - --oidc-client-id="": The client ID for the OpenID Connect client, must be set if oidc-issuer-url is set - --oidc-issuer-url="": The URL of the OpenID issuer, only HTTPS scheme will be accepted. If set, it will be used to verify the OIDC JSON Web Token (JWT) - --oidc-username-claim="sub": The OpenID claim to use as the user name. Note that claims other than the default ('sub') is not guaranteed to be unique and immutable. This flag is experimental, please see the authentication documentation for further details. - --profiling[=true]: Enable profiling via web interface host:port/debug/pprof/ - --runtime-config=: A set of key=value pairs that describe runtime configuration that may be passed to apiserver. apis/ key can be used to turn on/off specific api versions. apis// can be used to turn on/off specific resources. api/all and api/legacy are special keys to control all and legacy api versions respectively. - --secure-port=6443: The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. - --service-account-key-file="": File containing PEM-encoded x509 RSA private or public key, used to verify ServiceAccount tokens. If unspecified, --tls-private-key-file is used. - --service-account-lookup[=false]: If true, validate ServiceAccount tokens exist in etcd as part of authentication. - --service-cluster-ip-range=: A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes for pods. - --service-node-port-range=: A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range. - --ssh-keyfile="": If non-empty, use secure SSH proxy to the nodes, using this user keyfile - --ssh-user="": If non-empty, use secure SSH proxy to the nodes, using this user name - --storage-versions="extensions/v1beta1,v1": The versions to store resources with. Different groups may be stored in different versions. Specified in the format "group1/version1,group2/version2...". This flag expects a complete list of storage versions of ALL groups registered in the server. It defaults to a list of preferred versions of all registered groups, which is derived from the KUBE_API_VERSIONS environment variable. - --tls-cert-file="": File containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes. - --tls-private-key-file="": File containing x509 private key matching --tls-cert-file. - --token-auth-file="": If set, the file that will be used to secure the secure port of the API server via token authentication. - --watch-cache[=true]: Enable watch caching in the apiserver -{% endraw %} -``` - -###### Auto generated by spf13/cobra at 2015-10-29 20:12:33.554980405 +0000 UTC - - - diff --git a/v1.1/docs/admin/kube-controller-manager.md b/v1.1/docs/admin/kube-controller-manager.md deleted file mode 100644 index d852a4c3b7..0000000000 --- a/v1.1/docs/admin/kube-controller-manager.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -title: "kube-controller-manager" ---- - - -## kube-controller-manager - - - -### Synopsis - - -The Kubernetes controller manager is a daemon that embeds -the core control loops shipped with Kubernetes. In applications of robotics and -automation, a control loop is a non-terminating loop that regulates the state of -the system. In Kubernetes, a controller is a control loop that watches the shared -state of the cluster through the apiserver and makes changes attempting to move the -current state towards the desired state. Examples of controllers that ship with -Kubernetes today are the replication controller, endpoints controller, namespace -controller, and serviceaccounts controller. - -``` -{% raw %} -kube-controller-manager -{% endraw %} -``` - -### Options - -``` -{% raw %} - --address=127.0.0.1: The IP address to serve on (set to 0.0.0.0 for all interfaces) - --allocate-node-cidrs[=false]: Should CIDRs for Pods be allocated and set on the cloud provider. - --cloud-config="": The path to the cloud provider configuration file. Empty string for no configuration file. - --cloud-provider="": The provider for cloud services. Empty string for no provider. - --cluster-cidr=: CIDR Range for Pods in cluster. - --cluster-name="kubernetes": The instance prefix for the cluster - --concurrent-endpoint-syncs=5: The number of endpoint syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load - --concurrent_rc_syncs=5: The number of replication controllers that are allowed to sync concurrently. Larger number = more reponsive replica management, but more CPU (and network) load - --deleting-pods-burst=10: Number of nodes on which pods are bursty deleted in case of node failure. For more details look into RateLimiter. - --deleting-pods-qps=0.1: Number of nodes per second on which pods are deleted in case of node failure. - --deployment-controller-sync-period=30s: Period for syncing the deployments. - --google-json-key="": The Google Cloud Platform Service Account JSON Key to use for authentication. - --horizontal-pod-autoscaler-sync-period=30s: The period for syncing the number of pods in horizontal pod autoscaler. - --kubeconfig="": Path to kubeconfig file with authorization and master location information. - --log-flush-frequency=5s: Maximum number of seconds between log flushes - --master="": The address of the Kubernetes API server (overrides any value in kubeconfig) - --min-resync-period=12h0m0s: The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod - --namespace-sync-period=5m0s: The period for syncing namespace life-cycle updates - --node-monitor-grace-period=40s: Amount of time which we allow running Node to be unresponsive before marking it unhealty. Must be N times more than kubelet's nodeStatusUpdateFrequency, where N means number of retries allowed for kubelet to post node status. - --node-monitor-period=5s: The period for syncing NodeStatus in NodeController. - --node-startup-grace-period=1m0s: Amount of time which we allow starting Node to be unresponsive before marking it unhealty. - --node-sync-period=10s: The period for syncing nodes from cloudprovider. Longer periods will result in fewer calls to cloud provider, but may delay addition of new nodes to cluster. - --pod-eviction-timeout=5m0s: The grace period for deleting pods on failed nodes. - --port=10252: The port that the controller-manager's http service runs on - --profiling[=true]: Enable profiling via web interface host:port/debug/pprof/ - --pv-recycler-increment-timeout-nfs=30: the increment of time added per Gi to ActiveDeadlineSeconds for an NFS scrubber pod - --pv-recycler-minimum-timeout-hostpath=60: The minimum ActiveDeadlineSeconds to use for a HostPath Recycler pod. This is for development and testing only and will not work in a multi-node cluster. - --pv-recycler-minimum-timeout-nfs=300: The minimum ActiveDeadlineSeconds to use for an NFS Recycler pod - --pv-recycler-pod-template-filepath-hostpath="": The file path to a pod definition used as a template for HostPath persistent volume recycling. This is for development and testing only and will not work in a multi-node cluster. - --pv-recycler-pod-template-filepath-nfs="": The file path to a pod definition used as a template for NFS persistent volume recycling - --pv-recycler-timeout-increment-hostpath=30: the increment of time added per Gi to ActiveDeadlineSeconds for a HostPath scrubber pod. This is for development and testing only and will not work in a multi-node cluster. - --pvclaimbinder-sync-period=10s: The period for syncing persistent volumes and persistent volume claims - --resource-quota-sync-period=10s: The period for syncing quota usage status in the system - --root-ca-file="": If set, this root certificate authority will be included in service account's token secret. This must be a valid PEM-encoded CA bundle. - --service-account-private-key-file="": Filename containing a PEM-encoded private RSA key used to sign service account tokens. - --service-sync-period=5m0s: The period for syncing services with their external load balancers - --terminated-pod-gc-threshold=12500: Number of terminated pods that can exist before the terminated pod garbage collector starts deleting terminated pods. If <= 0, the terminated pod garbage collector is disabled. -{% endraw %} -``` - -###### Auto generated by spf13/cobra at 2015-10-29 20:12:25.539938496 +0000 UTC - - - diff --git a/v1.1/docs/admin/kube-proxy.md b/v1.1/docs/admin/kube-proxy.md deleted file mode 100644 index ec283ee56b..0000000000 --- a/v1.1/docs/admin/kube-proxy.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "kube-proxy" ---- - - -## kube-proxy - - - -### Synopsis - - -The Kubernetes network proxy runs on each node. This -reflects services as defined in the Kubernetes API on each node and can do simple -TCP,UDP stream forwarding or round robin TCP,UDP forwarding across a set of backends. -Service cluster ips and ports are currently found through Docker-links-compatible -environment variables specifying ports opened by the service proxy. There is an optional -addon that provides cluster DNS for these cluster IPs. The user must create a service -with the apiserver API to configure the proxy. - -``` -{% raw %} -kube-proxy -{% endraw %} -``` - -### Options - -``` -{% raw %} - --bind-address=0.0.0.0: The IP address for the proxy server to serve on (set to 0.0.0.0 for all interfaces) - --cleanup-iptables[=false]: If true cleanup iptables rules and exit. - --google-json-key="": The Google Cloud Platform Service Account JSON Key to use for authentication. - --healthz-bind-address=127.0.0.1: The IP address for the health check server to serve on, defaulting to 127.0.0.1 (set to 0.0.0.0 for all interfaces) - --healthz-port=10249: The port to bind the health check server. Use 0 to disable. - --hostname-override="": If non-empty, will use this string as identification instead of the actual hostname. - --iptables-sync-period=30s: How often iptables rules are refreshed (e.g. '5s', '1m', '2h22m'). Must be greater than 0. - --kubeconfig="": Path to kubeconfig file with authorization information (the master location is set by the master flag). - --log-flush-frequency=5s: Maximum number of seconds between log flushes - --masquerade-all[=false]: If using the pure iptables proxy, SNAT everything - --master="": The address of the Kubernetes API server (overrides any value in kubeconfig) - --oom-score-adj=-999: The oom-score-adj value for kube-proxy process. Values must be within the range [-1000, 1000] - --proxy-mode="": Which proxy mode to use: 'userspace' (older, stable) or 'iptables' (experimental). If blank, look at the Node object on the Kubernetes API and respect the 'net.experimental.kubernetes.io/proxy-mode' annotation if provided. Otherwise use the best-available proxy (currently userspace, but may change in future versions). If the iptables proxy is selected, regardless of how, but the system's kernel or iptables versions are insufficient, this always falls back to the userspace proxy. - --proxy-port-range=: Range of host ports (beginPort-endPort, inclusive) that may be consumed in order to proxy service traffic. If unspecified (0-0) then ports will be randomly chosen. - --resource-container="/kube-proxy": Absolute name of the resource-only container to create and run the Kube-proxy in (Default: /kube-proxy). - --udp-timeout=250ms: How long an idle UDP connection will be kept open (e.g. '250ms', '2s'). Must be greater than 0. Only applicable for proxy-mode=userspace -{% endraw %} -``` - -###### Auto generated by spf13/cobra at 2015-10-29 20:12:28.465584706 +0000 UTC - - - diff --git a/v1.1/docs/admin/kube-scheduler.md b/v1.1/docs/admin/kube-scheduler.md deleted file mode 100644 index b9079ce7bf..0000000000 --- a/v1.1/docs/admin/kube-scheduler.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -title: "kube-scheduler" ---- - - -## kube-scheduler - - - -### Synopsis - - -The Kubernetes scheduler is a policy-rich, topology-aware, -workload-specific function that significantly impacts availability, performance, -and capacity. The scheduler needs to take into account individual and collective -resource requirements, quality of service requirements, hardware/software/policy -constraints, affinity and anti-affinity specifications, data locality, inter-workload -interference, deadlines, and so on. Workload-specific requirements will be exposed -through the API as necessary. - -``` -{% raw %} -kube-scheduler -{% endraw %} -``` - -### Options - -``` -{% raw %} - --address=127.0.0.1: The IP address to serve on (set to 0.0.0.0 for all interfaces) - --algorithm-provider="DefaultProvider": The scheduling algorithm provider to use, one of: DefaultProvider - --bind-pods-burst=100: Number of bindings per second scheduler is allowed to make during bursts - --bind-pods-qps=50: Number of bindings per second scheduler is allowed to continuously make - --google-json-key="": The Google Cloud Platform Service Account JSON Key to use for authentication. - --kubeconfig="": Path to kubeconfig file with authorization and master location information. - --log-flush-frequency=5s: Maximum number of seconds between log flushes - --master="": The address of the Kubernetes API server (overrides any value in kubeconfig) - --policy-config-file="": File with scheduler policy configuration - --port=10251: The port that the scheduler's http service runs on - --profiling[=true]: Enable profiling via web interface host:port/debug/pprof/ -{% endraw %} -``` - -###### Auto generated by spf13/cobra at 2015-10-29 20:12:20.542446971 +0000 UTC - - - diff --git a/v1.1/docs/admin/kubelet.md b/v1.1/docs/admin/kubelet.md deleted file mode 100644 index b17d642632..0000000000 --- a/v1.1/docs/admin/kubelet.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -title: "kubelet" ---- - - -## kubelet - - - -### Synopsis - - -The kubelet is the primary "node agent" that runs on each -node. The kubelet works in terms of a PodSpec. A PodSpec is a YAML or JSON object -that describes a pod. The kubelet takes a set of PodSpecs that are provided through -various mechanisms (primarily through the apiserver) and ensures that the containers -described in those PodSpecs are running and healthy. - -Other than from an PodSpec from the apiserver, there are three ways that a container -manifest can be provided to the Kubelet. - -File: Path passed as a flag on the command line. This file is rechecked every 20 -seconds (configurable with a flag). - -HTTP endpoint: HTTP endpoint passed as a parameter on the command line. This endpoint -is checked every 20 seconds (also configurable with a flag). - -HTTP server: The kubelet can also listen for HTTP and respond to a simple API -(underspec'd currently) to submit a new manifest. - -``` -{% raw %} -kubelet -{% endraw %} -``` - -### Options - -``` -{% raw %} - --address=0.0.0.0: The IP address for the Kubelet to serve on (set to 0.0.0.0 for all interfaces) - --allow-privileged[=false]: If true, allow containers to request privileged mode. [default=false] - --api-servers=[]: List of Kubernetes API servers for publishing events, and reading pods and services. (ip:port), comma separated. - --cadvisor-port=4194: The port of the localhost cAdvisor endpoint - --cert-dir="/var/run/kubernetes": The directory where the TLS certs are located (by default /var/run/kubernetes). If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. - --cgroup-root="": Optional root cgroup to use for pods. This is handled by the container runtime on a best effort basis. Default: '', which means use the container runtime default. - --chaos-chance=0: If > 0.0, introduce random client errors and latency. Intended for testing. [default=0.0] - --cloud-config="": The path to the cloud provider configuration file. Empty string for no configuration file. - --cloud-provider="": The provider for cloud services. Empty string for no provider. - --cluster-dns=: IP address for a cluster DNS server. If set, kubelet will configure all containers to use this for DNS resolution in addition to the host's DNS servers - --cluster-domain="": Domain for this cluster. If set, kubelet will configure all containers to search this domain in addition to the host's search domains - --config="": Path to the config file or directory of files - --configure-cbr0[=false]: If true, kubelet will configure cbr0 based on Node.Spec.PodCIDR. - --container-runtime="docker": The container runtime to use. Possible values: 'docker', 'rkt'. Default: 'docker'. - --containerized[=false]: Experimental support for running kubelet in a container. Intended for testing. [default=false] - --cpu-cfs-quota[=false]: Enable CPU CFS quota enforcement for containers that specify CPU limits - --docker-endpoint="": If non-empty, use this for the docker endpoint to communicate with - --docker-exec-handler="native": Handler to use when executing a command in a container. Valid values are 'native' and 'nsenter'. Defaults to 'native'. - --enable-debugging-handlers[=true]: Enables server endpoints for log collection and local running of containers and commands - --enable-server[=true]: Enable the Kubelet's server - --event-burst=0: Maximum size of a bursty event records, temporarily allows event records to burst to this number, while still not exceeding event-qps. Only used if --event-qps > 0 - --event-qps=0: If > 0, limit event creations per second to this value. If 0, unlimited. [default=0.0] - --file-check-frequency=20s: Duration between checking config files for new data - --google-json-key="": The Google Cloud Platform Service Account JSON Key to use for authentication. - --healthz-bind-address=127.0.0.1: The IP address for the healthz server to serve on, defaulting to 127.0.0.1 (set to 0.0.0.0 for all interfaces) - --healthz-port=10248: The port of the localhost healthz endpoint - --host-ipc-sources="*": Comma-separated list of sources from which the Kubelet allows pods to use the host ipc namespace. [default="*"] - --host-network-sources="*": Comma-separated list of sources from which the Kubelet allows pods to use of host network. [default="*"] - --host-pid-sources="*": Comma-separated list of sources from which the Kubelet allows pods to use the host pid namespace. [default="*"] - --hostname-override="": If non-empty, will use this string as identification instead of the actual hostname. - --http-check-frequency=20s: Duration between checking http for new data - --image-gc-high-threshold=90: The percent of disk usage after which image garbage collection is always run. Default: 90%% - --image-gc-low-threshold=80: The percent of disk usage before which image garbage collection is never run. Lowest disk usage to garbage collect to. Default: 80%% - --kubeconfig="/var/lib/kubelet/kubeconfig": Path to a kubeconfig file, specifying how to authenticate to API server (the master location is set by the api-servers flag). - --log-flush-frequency=5s: Maximum number of seconds between log flushes - --low-diskspace-threshold-mb=256: The absolute free disk space, in MB, to maintain. When disk space falls below this threshold, new pods would be rejected. Default: 256 - --manifest-url="": URL for accessing the container manifest - --manifest-url-header="": HTTP header to use when accessing the manifest URL, with the key separated from the value with a ':', as in 'key:value' - --master-service-namespace="default": The namespace from which the kubernetes master services should be injected into pods - --max-open-files=1000000: Number of files that can be opened by Kubelet process. [default=1000000] - --max-pods=40: Number of Pods that can run on this Kubelet. - --maximum-dead-containers=100: Maximum number of old instances of containers to retain globally. Each container takes up some disk space. Default: 100. - --maximum-dead-containers-per-container=2: Maximum number of old instances to retain per container. Each container takes up some disk space. Default: 2. - --minimum-container-ttl-duration=1m0s: Minimum age for a finished container before it is garbage collected. Examples: '300ms', '10s' or '2h45m' - --network-plugin="": The name of the network plugin to be invoked for various events in kubelet/pod lifecycle - --network-plugin-dir="/usr/libexec/kubernetes/kubelet-plugins/net/exec/": The full path of the directory in which to search for network plugins - --node-status-update-frequency=10s: Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. Default: 10s - --oom-score-adj=-999: The oom-score-adj value for kubelet process. Values must be within the range [-1000, 1000] - --pod-cidr="": The CIDR to use for pod IP addresses, only used in standalone mode. In cluster mode, this is obtained from the master. - --pod-infra-container-image="gcr.io/google_containers/pause:0.8.0": The image whose network/ipc namespaces containers in each pod will use. - --port=10250: The port for the Kubelet to serve on. Note that "kubectl logs" will not work if you set this flag. - --read-only-port=10255: The read-only port for the Kubelet to serve on (set to 0 to disable) - --really-crash-for-testing[=false]: If true, when panics occur crash. Intended for testing. - --register-node[=true]: Register the node with the apiserver (defaults to true if --api-servers is set) - --registry-burst=10: Maximum size of a bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding registry-qps. Only used if --registry-qps > 0 - --registry-qps=0: If > 0, limit registry pull QPS to this value. If 0, unlimited. [default=0.0] - --resolv-conf="/etc/resolv.conf": Resolver configuration file used as the basis for the container DNS resolution configuration. - --resource-container="/kubelet": Absolute name of the resource-only container to create and run the Kubelet in (Default: /kubelet). - --rkt-path="": Path of rkt binary. Leave empty to use the first rkt in $PATH. Only used if --container-runtime='rkt' - --rkt-stage1-image="": image to use as stage1. Local paths and http/https URLs are supported. If empty, the 'stage1.aci' in the same directory as '--rkt-path' will be used - --root-dir="/var/lib/kubelet": Directory path for managing kubelet files (volume mounts,etc). - --runonce[=false]: If true, exit after spawning pods from local manifests or remote urls. Exclusive with --api-servers, and --enable-server - --serialize-image-pulls[=true]: Pull images one at a time. We recommend *not* changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details. [default=true] - --streaming-connection-idle-timeout=0: Maximum time a streaming connection can be idle before the connection is automatically closed. Example: '5m' - --sync-frequency=10s: Max period between synchronizing running containers and config - --system-container="": Optional resource-only container in which to place all non-kernel processes that are not already in a container. Empty for no container. Rolling back the flag requires a reboot. (Default: ""). - --tls-cert-file="": File containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to --cert-dir. - --tls-private-key-file="": File containing x509 private key matching --tls-cert-file. -{% endraw %} -``` - -###### Auto generated by spf13/cobra at 2015-10-29 20:12:15.480131233 +0000 UTC - - - diff --git a/v1.1/docs/admin/limitrange/README.md b/v1.1/docs/admin/limitrange/README.md index b63affb919..8bb060dbd3 100644 --- a/v1.1/docs/admin/limitrange/README.md +++ b/v1.1/docs/admin/limitrange/README.md @@ -1,222 +1,202 @@ ---- -title: "Limit Range" ---- - -Limit Range -======================================== -By default, pods run with unbounded CPU and memory limits. This means that any pod in the -system will be able to consume as much CPU and memory on the node that executes the pod. - -Users may want to impose restrictions on the amount of resource a single pod in the system may consume -for a variety of reasons. - -For example: - -1. Each node in the cluster has 2GB of memory. The cluster operator does not want to accept pods -that require more than 2GB of memory since no node in the cluster can support the requirement. To prevent a -pod from being permanently unscheduled to a node, the operator instead chooses to reject pods that exceed 2GB -of memory as part of admission control. -2. A cluster is shared by two communities in an organization that runs production and development workloads -respectively. Production workloads may consume up to 8GB of memory, but development workloads may consume up -to 512MB of memory. The cluster operator creates a separate namespace for each workload, and applies limits to -each namespace. -3. Users may create a pod which consumes resources just below the capacity of a machine. The left over space -may be too small to be useful, but big enough for the waste to be costly over the entire cluster. As a result, -the cluster operator may want to set limits that a pod must consume at least 20% of the memory and cpu of their -average node size in order to provide for more uniform scheduling and to limit waste. - -This example demonstrates how limits can be applied to a Kubernetes namespace to control -min/max resource limits per pod. In addition, this example demonstrates how you can -apply default resource limits to pods in the absence of an end-user specified value. - -See [LimitRange design doc](../../design/admission_control_limit_range.html) for more information. For a detailed description of the Kubernetes resource model, see [Resources](../../../docs/user-guide/compute-resources.html) - -Step 0: Prerequisites ------------------------------------------ -This example requires a running Kubernetes cluster. See the [Getting Started guides](../../../docs/getting-started-guides/) for how to get started. - -Change to the `` directory if you're not already there. - -Step 1: Create a namespace ------------------------------------------ -This example will work in a custom namespace to demonstrate the concepts involved. - -Let's create a new namespace called limit-example: - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/limitrange/namespace.yaml -namespace "limit-example" created -$ kubectl get namespaces -NAME LABELS STATUS AGE -default Active 5m -limit-example Active 53s -{% endraw %} -{% endhighlight %} - -Step 2: Apply a limit to the namespace ------------------------------------------ -Let's create a simple limit in our namespace. - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/limitrange/limits.yaml --namespace=limit-example -limitrange "mylimits" created -{% endraw %} -{% endhighlight %} - -Let's describe the limits that we have imposed in our namespace. - -{% highlight console %} -{% raw %} -$ kubectl describe limits mylimits --namespace=limit-example -Name: mylimits -Namespace: limit-example -Type Resource Min Max Request Limit Limit/Request ----- -------- --- --- ------- ----- ------------- -Pod cpu 200m 2 - - - -Pod memory 6Mi 1Gi - - - -Container cpu 100m 2 200m 300m - -Container memory 3Mi 1Gi 100Mi 200Mi - -{% endraw %} -{% endhighlight %} - -In this scenario, we have said the following: - -1. If a max constraint is specified for a resource (2 CPU and 1Gi memory in this case), then a limit -must be specified for that resource across all containers. Failure to specify a limit will result in -a validation error when attempting to create the pod. Note that a default value of limit is set by -*default* in file `limits.yaml` (300m CPU and 200Mi memory). -2. If a min constraint is specified for a resource (100m CPU and 3Mi memory in this case), then a -request must be specified for that resource across all containers. Failure to specify a request will -result in a validation error when attempting to create the pod. Note that a default value of request is -set by *defaultRequest* in file `limits.yaml` (200m CPU and 100Mi memory). -3. For any pod, the sum of all containers memory requests must be >= 6Mi and the sum of all containers -memory limits must be <= 1Gi; the sum of all containers CPU requests must be >= 200m and the sum of all -containers CPU limits must be <= 2. - -Step 3: Enforcing limits at point of creation ------------------------------------------ -The limits enumerated in a namespace are only enforced when a pod is created or updated in -the cluster. If you change the limits to a different value range, it does not affect pods that -were previously created in a namespace. - -If a resource (cpu or memory) is being restricted by a limit, the user will get an error at time -of creation explaining why. - -Let's first spin up a replication controller that creates a single container pod to demonstrate -how default values are applied to each pod. - -{% highlight console %} -{% raw %} -$ kubectl run nginx --image=nginx --replicas=1 --namespace=limit-example -replicationcontroller "nginx" created -$ kubectl get pods --namespace=limit-example -NAME READY STATUS RESTARTS AGE -nginx-aq0mf 1/1 Running 0 35s -$ kubectl get pods nginx-aq0mf --namespace=limit-example -o yaml | grep resources -C 8 -{% endraw %} -{% endhighlight %} - -{% highlight yaml %} -{% raw %} - resourceVersion: "127" - selfLink: /api/v1/namespaces/limit-example/pods/nginx-aq0mf - uid: 51be42a7-7156-11e5-9921-286ed488f785 -spec: - containers: - - image: nginx - imagePullPolicy: IfNotPresent - name: nginx - resources: - limits: - cpu: 300m - memory: 200Mi - requests: - cpu: 200m - memory: 100Mi - terminationMessagePath: /dev/termination-log - volumeMounts: -{% endraw %} -{% endhighlight %} - -Note that our nginx container has picked up the namespace default cpu and memory resource *limits* and *requests*. - -Let's create a pod that exceeds our allowed limits by having it have a container that requests 3 cpu cores. - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/limitrange/invalid-pod.yaml --namespace=limit-example -Error from server: error when creating "docs/admin/limitrange/invalid-pod.yaml": Pod "invalid-pod" is forbidden: [Maximum cpu usage per Pod is 2, but limit is 3., Maximum cpu usage per Container is 2, but limit is 3.] -{% endraw %} -{% endhighlight %} - -Let's create a pod that falls within the allowed limit boundaries. - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/limitrange/valid-pod.yaml --namespace=limit-example -pod "valid-pod" created -$ kubectl get pods valid-pod --namespace=limit-example -o yaml | grep -C 6 resources -{% endraw %} -{% endhighlight %} - -{% highlight yaml %} -{% raw %} - uid: 162a12aa-7157-11e5-9921-286ed488f785 -spec: - containers: - - image: gcr.io/google_containers/serve_hostname - imagePullPolicy: IfNotPresent - name: kubernetes-serve-hostname - resources: - limits: - cpu: "1" - memory: 512Mi - requests: - cpu: "1" - memory: 512Mi -{% endraw %} -{% endhighlight %} - -Note that this pod specifies explicit resource *limits* and *requests* so it did not pick up the namespace -default values. - -Note: The *limits* for CPU resource are not enforced in the default Kubernetes setup on the physical node -that runs the container unless the administrator deploys the kubelet with the folllowing flag: - -``` -{% raw %} -$ kubelet --help -Usage of kubelet -.... - --cpu-cfs-quota[=false]: Enable CPU CFS quota enforcement for containers that specify CPU limits -$ kubelet --cpu-cfs-quota=true ... -{% endraw %} -``` - -Step 4: Cleanup ----------------------------- -To remove the resources used by this example, you can just delete the limit-example namespace. - -{% highlight console %} -{% raw %} -$ kubectl delete namespace limit-example -namespace "limit-example" deleted -$ kubectl get namespaces -NAME LABELS STATUS AGE -default Active 20m -{% endraw %} -{% endhighlight %} - -Summary ----------------------------- -Cluster operators that want to restrict the amount of resources a single container or pod may consume -are able to define allowable ranges per Kubernetes namespace. In the absence of any explicit assignments, -the Kubernetes system is able to apply default resource *limits* and *requests* if desired in order to -constrain the amount of resource a pod consumes on a node. - - - - - +--- +title: "Limit Range" +--- + +Limit Range +======================================== +By default, pods run with unbounded CPU and memory limits. This means that any pod in the +system will be able to consume as much CPU and memory on the node that executes the pod. + +Users may want to impose restrictions on the amount of resource a single pod in the system may consume +for a variety of reasons. + +For example: + +1. Each node in the cluster has 2GB of memory. The cluster operator does not want to accept pods +that require more than 2GB of memory since no node in the cluster can support the requirement. To prevent a +pod from being permanently unscheduled to a node, the operator instead chooses to reject pods that exceed 2GB +of memory as part of admission control. +2. A cluster is shared by two communities in an organization that runs production and development workloads +respectively. Production workloads may consume up to 8GB of memory, but development workloads may consume up +to 512MB of memory. The cluster operator creates a separate namespace for each workload, and applies limits to +each namespace. +3. Users may create a pod which consumes resources just below the capacity of a machine. The left over space +may be too small to be useful, but big enough for the waste to be costly over the entire cluster. As a result, +the cluster operator may want to set limits that a pod must consume at least 20% of the memory and cpu of their +average node size in order to provide for more uniform scheduling and to limit waste. + +This example demonstrates how limits can be applied to a Kubernetes namespace to control +min/max resource limits per pod. In addition, this example demonstrates how you can +apply default resource limits to pods in the absence of an end-user specified value. + +See [LimitRange design doc](../../design/admission_control_limit_range) for more information. For a detailed description of the Kubernetes resource model, see [Resources](/{{page.version}}/docs/user-guide/compute-resources) + +Step 0: Prerequisites +----------------------------------------- +This example requires a running Kubernetes cluster. See the [Getting Started guides](/{{page.version}}/docs/getting-started-guides/) for how to get started. + +Change to the `` directory if you're not already there. + +Step 1: Create a namespace +----------------------------------------- +This example will work in a custom namespace to demonstrate the concepts involved. + +Let's create a new namespace called limit-example: + +{% highlight console %} +$ kubectl create -f docs/admin/limitrange/namespace.yaml +namespace "limit-example" created +$ kubectl get namespaces +NAME LABELS STATUS AGE +default Active 5m +limit-example Active 53s +{% endhighlight %} + +Step 2: Apply a limit to the namespace +----------------------------------------- +Let's create a simple limit in our namespace. + +{% highlight console %} +$ kubectl create -f docs/admin/limitrange/limits.yaml --namespace=limit-example +limitrange "mylimits" created +{% endhighlight %} + +Let's describe the limits that we have imposed in our namespace. + +{% highlight console %} +$ kubectl describe limits mylimits --namespace=limit-example +Name: mylimits +Namespace: limit-example +Type Resource Min Max Request Limit Limit/Request +---- -------- --- --- ------- ----- ------------- +Pod cpu 200m 2 - - - +Pod memory 6Mi 1Gi - - - +Container cpu 100m 2 200m 300m - +Container memory 3Mi 1Gi 100Mi 200Mi - +{% endhighlight %} + +In this scenario, we have said the following: + +1. If a max constraint is specified for a resource (2 CPU and 1Gi memory in this case), then a limit +must be specified for that resource across all containers. Failure to specify a limit will result in +a validation error when attempting to create the pod. Note that a default value of limit is set by +*default* in file `limits.yaml` (300m CPU and 200Mi memory). +2. If a min constraint is specified for a resource (100m CPU and 3Mi memory in this case), then a +request must be specified for that resource across all containers. Failure to specify a request will +result in a validation error when attempting to create the pod. Note that a default value of request is +set by *defaultRequest* in file `limits.yaml` (200m CPU and 100Mi memory). +3. For any pod, the sum of all containers memory requests must be >= 6Mi and the sum of all containers +memory limits must be <= 1Gi; the sum of all containers CPU requests must be >= 200m and the sum of all +containers CPU limits must be <= 2. + +Step 3: Enforcing limits at point of creation +----------------------------------------- +The limits enumerated in a namespace are only enforced when a pod is created or updated in +the cluster. If you change the limits to a different value range, it does not affect pods that +were previously created in a namespace. + +If a resource (cpu or memory) is being restricted by a limit, the user will get an error at time +of creation explaining why. + +Let's first spin up a replication controller that creates a single container pod to demonstrate +how default values are applied to each pod. + +{% highlight console %} +$ kubectl run nginx --image=nginx --replicas=1 --namespace=limit-example +replicationcontroller "nginx" created +$ kubectl get pods --namespace=limit-example +NAME READY STATUS RESTARTS AGE +nginx-aq0mf 1/1 Running 0 35s +$ kubectl get pods nginx-aq0mf --namespace=limit-example -o yaml | grep resources -C 8 +{% endhighlight %} + +{% highlight yaml %} + resourceVersion: "127" + selfLink: /api/v1/namespaces/limit-example/pods/nginx-aq0mf + uid: 51be42a7-7156-11e5-9921-286ed488f785 +spec: + containers: + - image: nginx + imagePullPolicy: IfNotPresent + name: nginx + resources: + limits: + cpu: 300m + memory: 200Mi + requests: + cpu: 200m + memory: 100Mi + terminationMessagePath: /dev/termination-log + volumeMounts: +{% endhighlight %} + +Note that our nginx container has picked up the namespace default cpu and memory resource *limits* and *requests*. + +Let's create a pod that exceeds our allowed limits by having it have a container that requests 3 cpu cores. + +{% highlight console %} +$ kubectl create -f docs/admin/limitrange/invalid-pod.yaml --namespace=limit-example +Error from server: error when creating "docs/admin/limitrange/invalid-pod.yaml": Pod "invalid-pod" is forbidden: [Maximum cpu usage per Pod is 2, but limit is 3., Maximum cpu usage per Container is 2, but limit is 3.] +{% endhighlight %} + +Let's create a pod that falls within the allowed limit boundaries. + +{% highlight console %} +$ kubectl create -f docs/admin/limitrange/valid-pod.yaml --namespace=limit-example +pod "valid-pod" created +$ kubectl get pods valid-pod --namespace=limit-example -o yaml | grep -C 6 resources +{% endhighlight %} + +{% highlight yaml %} + uid: 162a12aa-7157-11e5-9921-286ed488f785 +spec: + containers: + - image: gcr.io/google_containers/serve_hostname + imagePullPolicy: IfNotPresent + name: kubernetes-serve-hostname + resources: + limits: + cpu: "1" + memory: 512Mi + requests: + cpu: "1" + memory: 512Mi +{% endhighlight %} + +Note that this pod specifies explicit resource *limits* and *requests* so it did not pick up the namespace +default values. + +Note: The *limits* for CPU resource are not enforced in the default Kubernetes setup on the physical node +that runs the container unless the administrator deploys the kubelet with the folllowing flag: + +``` +$ kubelet --help +Usage of kubelet +.... + --cpu-cfs-quota[=false]: Enable CPU CFS quota enforcement for containers that specify CPU limits +$ kubelet --cpu-cfs-quota=true ... +``` + +Step 4: Cleanup +---------------------------- +To remove the resources used by this example, you can just delete the limit-example namespace. + +{% highlight console %} +$ kubectl delete namespace limit-example +namespace "limit-example" deleted +$ kubectl get namespaces +NAME LABELS STATUS AGE +default Active 20m +{% endhighlight %} + +Summary +---------------------------- +Cluster operators that want to restrict the amount of resources a single container or pod may consume +are able to define allowable ranges per Kubernetes namespace. In the absence of any explicit assignments, +the Kubernetes system is able to apply default resource *limits* and *requests* if desired in order to +constrain the amount of resource a pod consumes on a node. + + + + + diff --git a/v1.1/docs/admin/limitrange/index.md b/v1.1/docs/admin/limitrange/index.md index b63affb919..8bb060dbd3 100644 --- a/v1.1/docs/admin/limitrange/index.md +++ b/v1.1/docs/admin/limitrange/index.md @@ -1,222 +1,202 @@ ---- -title: "Limit Range" ---- - -Limit Range -======================================== -By default, pods run with unbounded CPU and memory limits. This means that any pod in the -system will be able to consume as much CPU and memory on the node that executes the pod. - -Users may want to impose restrictions on the amount of resource a single pod in the system may consume -for a variety of reasons. - -For example: - -1. Each node in the cluster has 2GB of memory. The cluster operator does not want to accept pods -that require more than 2GB of memory since no node in the cluster can support the requirement. To prevent a -pod from being permanently unscheduled to a node, the operator instead chooses to reject pods that exceed 2GB -of memory as part of admission control. -2. A cluster is shared by two communities in an organization that runs production and development workloads -respectively. Production workloads may consume up to 8GB of memory, but development workloads may consume up -to 512MB of memory. The cluster operator creates a separate namespace for each workload, and applies limits to -each namespace. -3. Users may create a pod which consumes resources just below the capacity of a machine. The left over space -may be too small to be useful, but big enough for the waste to be costly over the entire cluster. As a result, -the cluster operator may want to set limits that a pod must consume at least 20% of the memory and cpu of their -average node size in order to provide for more uniform scheduling and to limit waste. - -This example demonstrates how limits can be applied to a Kubernetes namespace to control -min/max resource limits per pod. In addition, this example demonstrates how you can -apply default resource limits to pods in the absence of an end-user specified value. - -See [LimitRange design doc](../../design/admission_control_limit_range.html) for more information. For a detailed description of the Kubernetes resource model, see [Resources](../../../docs/user-guide/compute-resources.html) - -Step 0: Prerequisites ------------------------------------------ -This example requires a running Kubernetes cluster. See the [Getting Started guides](../../../docs/getting-started-guides/) for how to get started. - -Change to the `` directory if you're not already there. - -Step 1: Create a namespace ------------------------------------------ -This example will work in a custom namespace to demonstrate the concepts involved. - -Let's create a new namespace called limit-example: - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/limitrange/namespace.yaml -namespace "limit-example" created -$ kubectl get namespaces -NAME LABELS STATUS AGE -default Active 5m -limit-example Active 53s -{% endraw %} -{% endhighlight %} - -Step 2: Apply a limit to the namespace ------------------------------------------ -Let's create a simple limit in our namespace. - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/limitrange/limits.yaml --namespace=limit-example -limitrange "mylimits" created -{% endraw %} -{% endhighlight %} - -Let's describe the limits that we have imposed in our namespace. - -{% highlight console %} -{% raw %} -$ kubectl describe limits mylimits --namespace=limit-example -Name: mylimits -Namespace: limit-example -Type Resource Min Max Request Limit Limit/Request ----- -------- --- --- ------- ----- ------------- -Pod cpu 200m 2 - - - -Pod memory 6Mi 1Gi - - - -Container cpu 100m 2 200m 300m - -Container memory 3Mi 1Gi 100Mi 200Mi - -{% endraw %} -{% endhighlight %} - -In this scenario, we have said the following: - -1. If a max constraint is specified for a resource (2 CPU and 1Gi memory in this case), then a limit -must be specified for that resource across all containers. Failure to specify a limit will result in -a validation error when attempting to create the pod. Note that a default value of limit is set by -*default* in file `limits.yaml` (300m CPU and 200Mi memory). -2. If a min constraint is specified for a resource (100m CPU and 3Mi memory in this case), then a -request must be specified for that resource across all containers. Failure to specify a request will -result in a validation error when attempting to create the pod. Note that a default value of request is -set by *defaultRequest* in file `limits.yaml` (200m CPU and 100Mi memory). -3. For any pod, the sum of all containers memory requests must be >= 6Mi and the sum of all containers -memory limits must be <= 1Gi; the sum of all containers CPU requests must be >= 200m and the sum of all -containers CPU limits must be <= 2. - -Step 3: Enforcing limits at point of creation ------------------------------------------ -The limits enumerated in a namespace are only enforced when a pod is created or updated in -the cluster. If you change the limits to a different value range, it does not affect pods that -were previously created in a namespace. - -If a resource (cpu or memory) is being restricted by a limit, the user will get an error at time -of creation explaining why. - -Let's first spin up a replication controller that creates a single container pod to demonstrate -how default values are applied to each pod. - -{% highlight console %} -{% raw %} -$ kubectl run nginx --image=nginx --replicas=1 --namespace=limit-example -replicationcontroller "nginx" created -$ kubectl get pods --namespace=limit-example -NAME READY STATUS RESTARTS AGE -nginx-aq0mf 1/1 Running 0 35s -$ kubectl get pods nginx-aq0mf --namespace=limit-example -o yaml | grep resources -C 8 -{% endraw %} -{% endhighlight %} - -{% highlight yaml %} -{% raw %} - resourceVersion: "127" - selfLink: /api/v1/namespaces/limit-example/pods/nginx-aq0mf - uid: 51be42a7-7156-11e5-9921-286ed488f785 -spec: - containers: - - image: nginx - imagePullPolicy: IfNotPresent - name: nginx - resources: - limits: - cpu: 300m - memory: 200Mi - requests: - cpu: 200m - memory: 100Mi - terminationMessagePath: /dev/termination-log - volumeMounts: -{% endraw %} -{% endhighlight %} - -Note that our nginx container has picked up the namespace default cpu and memory resource *limits* and *requests*. - -Let's create a pod that exceeds our allowed limits by having it have a container that requests 3 cpu cores. - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/limitrange/invalid-pod.yaml --namespace=limit-example -Error from server: error when creating "docs/admin/limitrange/invalid-pod.yaml": Pod "invalid-pod" is forbidden: [Maximum cpu usage per Pod is 2, but limit is 3., Maximum cpu usage per Container is 2, but limit is 3.] -{% endraw %} -{% endhighlight %} - -Let's create a pod that falls within the allowed limit boundaries. - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/limitrange/valid-pod.yaml --namespace=limit-example -pod "valid-pod" created -$ kubectl get pods valid-pod --namespace=limit-example -o yaml | grep -C 6 resources -{% endraw %} -{% endhighlight %} - -{% highlight yaml %} -{% raw %} - uid: 162a12aa-7157-11e5-9921-286ed488f785 -spec: - containers: - - image: gcr.io/google_containers/serve_hostname - imagePullPolicy: IfNotPresent - name: kubernetes-serve-hostname - resources: - limits: - cpu: "1" - memory: 512Mi - requests: - cpu: "1" - memory: 512Mi -{% endraw %} -{% endhighlight %} - -Note that this pod specifies explicit resource *limits* and *requests* so it did not pick up the namespace -default values. - -Note: The *limits* for CPU resource are not enforced in the default Kubernetes setup on the physical node -that runs the container unless the administrator deploys the kubelet with the folllowing flag: - -``` -{% raw %} -$ kubelet --help -Usage of kubelet -.... - --cpu-cfs-quota[=false]: Enable CPU CFS quota enforcement for containers that specify CPU limits -$ kubelet --cpu-cfs-quota=true ... -{% endraw %} -``` - -Step 4: Cleanup ----------------------------- -To remove the resources used by this example, you can just delete the limit-example namespace. - -{% highlight console %} -{% raw %} -$ kubectl delete namespace limit-example -namespace "limit-example" deleted -$ kubectl get namespaces -NAME LABELS STATUS AGE -default Active 20m -{% endraw %} -{% endhighlight %} - -Summary ----------------------------- -Cluster operators that want to restrict the amount of resources a single container or pod may consume -are able to define allowable ranges per Kubernetes namespace. In the absence of any explicit assignments, -the Kubernetes system is able to apply default resource *limits* and *requests* if desired in order to -constrain the amount of resource a pod consumes on a node. - - - - - +--- +title: "Limit Range" +--- + +Limit Range +======================================== +By default, pods run with unbounded CPU and memory limits. This means that any pod in the +system will be able to consume as much CPU and memory on the node that executes the pod. + +Users may want to impose restrictions on the amount of resource a single pod in the system may consume +for a variety of reasons. + +For example: + +1. Each node in the cluster has 2GB of memory. The cluster operator does not want to accept pods +that require more than 2GB of memory since no node in the cluster can support the requirement. To prevent a +pod from being permanently unscheduled to a node, the operator instead chooses to reject pods that exceed 2GB +of memory as part of admission control. +2. A cluster is shared by two communities in an organization that runs production and development workloads +respectively. Production workloads may consume up to 8GB of memory, but development workloads may consume up +to 512MB of memory. The cluster operator creates a separate namespace for each workload, and applies limits to +each namespace. +3. Users may create a pod which consumes resources just below the capacity of a machine. The left over space +may be too small to be useful, but big enough for the waste to be costly over the entire cluster. As a result, +the cluster operator may want to set limits that a pod must consume at least 20% of the memory and cpu of their +average node size in order to provide for more uniform scheduling and to limit waste. + +This example demonstrates how limits can be applied to a Kubernetes namespace to control +min/max resource limits per pod. In addition, this example demonstrates how you can +apply default resource limits to pods in the absence of an end-user specified value. + +See [LimitRange design doc](../../design/admission_control_limit_range) for more information. For a detailed description of the Kubernetes resource model, see [Resources](/{{page.version}}/docs/user-guide/compute-resources) + +Step 0: Prerequisites +----------------------------------------- +This example requires a running Kubernetes cluster. See the [Getting Started guides](/{{page.version}}/docs/getting-started-guides/) for how to get started. + +Change to the `` directory if you're not already there. + +Step 1: Create a namespace +----------------------------------------- +This example will work in a custom namespace to demonstrate the concepts involved. + +Let's create a new namespace called limit-example: + +{% highlight console %} +$ kubectl create -f docs/admin/limitrange/namespace.yaml +namespace "limit-example" created +$ kubectl get namespaces +NAME LABELS STATUS AGE +default Active 5m +limit-example Active 53s +{% endhighlight %} + +Step 2: Apply a limit to the namespace +----------------------------------------- +Let's create a simple limit in our namespace. + +{% highlight console %} +$ kubectl create -f docs/admin/limitrange/limits.yaml --namespace=limit-example +limitrange "mylimits" created +{% endhighlight %} + +Let's describe the limits that we have imposed in our namespace. + +{% highlight console %} +$ kubectl describe limits mylimits --namespace=limit-example +Name: mylimits +Namespace: limit-example +Type Resource Min Max Request Limit Limit/Request +---- -------- --- --- ------- ----- ------------- +Pod cpu 200m 2 - - - +Pod memory 6Mi 1Gi - - - +Container cpu 100m 2 200m 300m - +Container memory 3Mi 1Gi 100Mi 200Mi - +{% endhighlight %} + +In this scenario, we have said the following: + +1. If a max constraint is specified for a resource (2 CPU and 1Gi memory in this case), then a limit +must be specified for that resource across all containers. Failure to specify a limit will result in +a validation error when attempting to create the pod. Note that a default value of limit is set by +*default* in file `limits.yaml` (300m CPU and 200Mi memory). +2. If a min constraint is specified for a resource (100m CPU and 3Mi memory in this case), then a +request must be specified for that resource across all containers. Failure to specify a request will +result in a validation error when attempting to create the pod. Note that a default value of request is +set by *defaultRequest* in file `limits.yaml` (200m CPU and 100Mi memory). +3. For any pod, the sum of all containers memory requests must be >= 6Mi and the sum of all containers +memory limits must be <= 1Gi; the sum of all containers CPU requests must be >= 200m and the sum of all +containers CPU limits must be <= 2. + +Step 3: Enforcing limits at point of creation +----------------------------------------- +The limits enumerated in a namespace are only enforced when a pod is created or updated in +the cluster. If you change the limits to a different value range, it does not affect pods that +were previously created in a namespace. + +If a resource (cpu or memory) is being restricted by a limit, the user will get an error at time +of creation explaining why. + +Let's first spin up a replication controller that creates a single container pod to demonstrate +how default values are applied to each pod. + +{% highlight console %} +$ kubectl run nginx --image=nginx --replicas=1 --namespace=limit-example +replicationcontroller "nginx" created +$ kubectl get pods --namespace=limit-example +NAME READY STATUS RESTARTS AGE +nginx-aq0mf 1/1 Running 0 35s +$ kubectl get pods nginx-aq0mf --namespace=limit-example -o yaml | grep resources -C 8 +{% endhighlight %} + +{% highlight yaml %} + resourceVersion: "127" + selfLink: /api/v1/namespaces/limit-example/pods/nginx-aq0mf + uid: 51be42a7-7156-11e5-9921-286ed488f785 +spec: + containers: + - image: nginx + imagePullPolicy: IfNotPresent + name: nginx + resources: + limits: + cpu: 300m + memory: 200Mi + requests: + cpu: 200m + memory: 100Mi + terminationMessagePath: /dev/termination-log + volumeMounts: +{% endhighlight %} + +Note that our nginx container has picked up the namespace default cpu and memory resource *limits* and *requests*. + +Let's create a pod that exceeds our allowed limits by having it have a container that requests 3 cpu cores. + +{% highlight console %} +$ kubectl create -f docs/admin/limitrange/invalid-pod.yaml --namespace=limit-example +Error from server: error when creating "docs/admin/limitrange/invalid-pod.yaml": Pod "invalid-pod" is forbidden: [Maximum cpu usage per Pod is 2, but limit is 3., Maximum cpu usage per Container is 2, but limit is 3.] +{% endhighlight %} + +Let's create a pod that falls within the allowed limit boundaries. + +{% highlight console %} +$ kubectl create -f docs/admin/limitrange/valid-pod.yaml --namespace=limit-example +pod "valid-pod" created +$ kubectl get pods valid-pod --namespace=limit-example -o yaml | grep -C 6 resources +{% endhighlight %} + +{% highlight yaml %} + uid: 162a12aa-7157-11e5-9921-286ed488f785 +spec: + containers: + - image: gcr.io/google_containers/serve_hostname + imagePullPolicy: IfNotPresent + name: kubernetes-serve-hostname + resources: + limits: + cpu: "1" + memory: 512Mi + requests: + cpu: "1" + memory: 512Mi +{% endhighlight %} + +Note that this pod specifies explicit resource *limits* and *requests* so it did not pick up the namespace +default values. + +Note: The *limits* for CPU resource are not enforced in the default Kubernetes setup on the physical node +that runs the container unless the administrator deploys the kubelet with the folllowing flag: + +``` +$ kubelet --help +Usage of kubelet +.... + --cpu-cfs-quota[=false]: Enable CPU CFS quota enforcement for containers that specify CPU limits +$ kubelet --cpu-cfs-quota=true ... +``` + +Step 4: Cleanup +---------------------------- +To remove the resources used by this example, you can just delete the limit-example namespace. + +{% highlight console %} +$ kubectl delete namespace limit-example +namespace "limit-example" deleted +$ kubectl get namespaces +NAME LABELS STATUS AGE +default Active 20m +{% endhighlight %} + +Summary +---------------------------- +Cluster operators that want to restrict the amount of resources a single container or pod may consume +are able to define allowable ranges per Kubernetes namespace. In the absence of any explicit assignments, +the Kubernetes system is able to apply default resource *limits* and *requests* if desired in order to +constrain the amount of resource a pod consumes on a node. + + + + + diff --git a/v1.1/docs/admin/multi-cluster.md b/v1.1/docs/admin/multi-cluster.md index 2a790909b6..4c9d1600c9 100644 --- a/v1.1/docs/admin/multi-cluster.md +++ b/v1.1/docs/admin/multi-cluster.md @@ -1,6 +1,5 @@ --- title: "Considerations for running multiple Kubernetes clusters" -section: guides --- You may want to set up multiple Kubernetes clusters, both to @@ -9,13 +8,13 @@ This document describes some of the issues to consider when making a decision ab Note that at present, Kubernetes does not offer a mechanism to aggregate multiple clusters into a single virtual cluster. However, -we [plan to do this in the future](../proposals/federation.html). +we [plan to do this in the future](../proposals/federation). ## Scope of a single cluster On IaaS providers such as Google Compute Engine or Amazon Web Services, a VM exists in a [zone](https://cloud.google.com/compute/docs/zones) or [availability -zone](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). +zone](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones). We suggest that all the VMs in a Kubernetes cluster should be in the same availability zone, because: - compared to having a single global Kubernetes cluster, there are fewer single-points of failure - compared to a cluster that spans availability zones, it is easier to reason about the availability properties of a diff --git a/v1.1/docs/admin/namespaces.md b/v1.1/docs/admin/namespaces.md index 8fc325b98a..74dc28a6e2 100644 --- a/v1.1/docs/admin/namespaces.md +++ b/v1.1/docs/admin/namespaces.md @@ -1,10 +1,6 @@ --- title: "Namespaces" --- - - -# Namespaces - ## Abstract A Namespace is a mechanism to partition resources created by users into @@ -50,12 +46,12 @@ Look [here](namespaces/) for an in depth example of namespaces. You can list the current namespaces in a cluster using: {% highlight console %} -{% raw %} + $ kubectl get namespaces NAME LABELS STATUS default Active kube-system Active -{% endraw %} + {% endhighlight %} Kubernetes starts with two initial namespaces: @@ -65,15 +61,15 @@ Kubernetes starts with two initial namespaces: You can also get the summary of a specific namespace using: {% highlight console %} -{% raw %} + $ kubectl get namespaces -{% endraw %} + {% endhighlight %} Or you can get detailed information with: {% highlight console %} -{% raw %} + $ kubectl describe namespaces Name: default Labels: @@ -85,7 +81,7 @@ Resource Limits Type Resource Min Max Default ---- -------- --- --- --- Container cpu - - 100m -{% endraw %} + {% endhighlight %} Note that these details show both resource quota (if present) as well as resource limit ranges. @@ -96,7 +92,7 @@ to define *Hard* resource usage limits that a *Namespace* may consume. A limit range defines min/max constraints on the amount of resources a single entity can consume in a *Namespace*. -See [Admission control: Limit Range](../design/admission_control_limit_range.html) +See [Admission control: Limit Range](../design/admission_control_limit_range) A namespace can be in one of two phases: * `Active` the namespace is in use @@ -109,12 +105,12 @@ See the [design doc](../design/namespaces.html#phases) for more details. To create a new namespace, first create a new YAML file called `my-namespace.yaml` with the contents: {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Namespace metadata: name: -{% endraw %} + {% endhighlight %} Note that the name of your namespace must be a DNS compatible label. @@ -124,24 +120,24 @@ More information on the `finalizers` field can be found in the namespace [design Then run: {% highlight console %} -{% raw %} + $ kubectl create -f ./my-namespace.yaml -{% endraw %} + {% endhighlight %} ### Working in namespaces -See [Setting the namespace for a request](../../docs/user-guide/namespaces.html#setting-the-namespace-for-a-request) -and [Setting the namespace preference](../../docs/user-guide/namespaces.html#setting-the-namespace-preference). +See [Setting the namespace for a request](/{{page.version}}/docs/user-guide/namespaces.html#setting-the-namespace-for-a-request) +and [Setting the namespace preference](/{{page.version}}/docs/user-guide/namespaces.html#setting-the-namespace-preference). ### Deleting a namespace You can delete a namespace with {% highlight console %} -{% raw %} + $ kubectl delete namespaces -{% endraw %} + {% endhighlight %} **WARNING, this deletes _everything_ under the namespace!** @@ -150,7 +146,7 @@ This delete is asynchronous, so for a time you will see the namespace in the `Te ## Namespaces and DNS -When you create a [Service](../../docs/user-guide/services.html), it creates a corresponding [DNS entry](dns.html). +When you create a [Service](/{{page.version}}/docs/user-guide/services), it creates a corresponding [DNS entry](dns). This entry is of the form `..svc.cluster.local`, which means that if a container just uses `` it will resolve to the service which is local to a namespace. This is useful for using the same configuration across @@ -160,7 +156,7 @@ across namespaces, you need to use the fully qualified domain name (FQDN). ## Design Details of the design of namespaces in Kubernetes, including a [detailed example](../design/namespaces.html#example-openshift-origin-managing-a-kubernetes-namespace) -can be found in the [namespaces design doc](../design/namespaces.html) +can be found in the [namespaces design doc](../design/namespaces) diff --git a/v1.1/docs/admin/namespaces/README.md b/v1.1/docs/admin/namespaces/README.md index c43101e42f..18feca69e4 100644 --- a/v1.1/docs/admin/namespaces/README.md +++ b/v1.1/docs/admin/namespaces/README.md @@ -1,283 +1,252 @@ ---- -title: "Kubernetes Namespaces" -section: guides ---- - -Kubernetes _[namespaces](../../../docs/admin/namespaces.html)_ help different projects, teams, or customers to share a Kubernetes cluster. - -It does this by providing the following: - -1. A scope for [Names](../../user-guide/identifiers.html). -2. A mechanism to attach authorization and policy to a subsection of the cluster. - -Use of multiple namespaces is optional. - -This example demonstrates how to use Kubernetes namespaces to subdivide your cluster. - -### Step Zero: Prerequisites - -This example assumes the following: - -1. You have an [existing Kubernetes cluster](../../getting-started-guides/). -2. You have a basic understanding of Kubernetes _[pods](../../user-guide/pods.html)_, _[services](../../user-guide/services.html)_, and _[replication controllers](../../user-guide/replication-controller.html)_. - -### Step One: Understand the default namespace - -By default, a Kubernetes cluster will instantiate a default namespace when provisioning the cluster to hold the default set of pods, -services, and replication controllers used by the cluster. - -Assuming you have a fresh cluster, you can introspect the available namespace's by doing the following: - -{% highlight console %} -{% raw %} -$ kubectl get namespaces -NAME LABELS -default -{% endraw %} -{% endhighlight %} - -### Step Two: Create new namespaces - -For this exercise, we will create two additional Kubernetes namespaces to hold our content. - -Let's imagine a scenario where an organization is using a shared Kubernetes cluster for development and production use cases. - -The development team would like to maintain a space in the cluster where they can get a view on the list of pods, services, and replication controllers -they use to build and run their application. In this space, Kubernetes resources come and go, and the restrictions on who can or cannot modify resources -are relaxed to enable agile development. - -The operations team would like to maintain a space in the cluster where they can enforce strict procedures on who can or cannot manipulate the set of -pods, services, and replication controllers that run the production site. - -One pattern this organization could follow is to partition the Kubernetes cluster into two namespaces: development and production. - -Let's create two new namespaces to hold our work. - -Use the file [`namespace-dev.json`](namespace-dev.json) which describes a development namespace: - - - -{% highlight json %} -{% raw %} -{ - "kind": "Namespace", - "apiVersion": "v1", - "metadata": { - "name": "development", - "labels": { - "name": "development" - } - } -} -{% endraw %} -{% endhighlight %} - -[Download example](namespace-dev.json) - - -Create the development namespace using kubectl. - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/namespaces/namespace-dev.json -{% endraw %} -{% endhighlight %} - -And then lets create the production namespace using kubectl. - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/namespaces/namespace-prod.json -{% endraw %} -{% endhighlight %} - -To be sure things are right, let's list all of the namespaces in our cluster. - -{% highlight console %} -{% raw %} -$ kubectl get namespaces -NAME LABELS STATUS -default Active -development name=development Active -production name=production Active -{% endraw %} -{% endhighlight %} - - -### Step Three: Create pods in each namespace - -A Kubernetes namespace provides the scope for pods, services, and replication controllers in the cluster. - -Users interacting with one namespace do not see the content in another namespace. - -To demonstrate this, let's spin up a simple replication controller and pod in the development namespace. - -We first check what is the current context: - -{% highlight yaml %} -{% raw %} -apiVersion: v1 -clusters: -- cluster: - certificate-authority-data: REDACTED - server: https://130.211.122.180 - name: lithe-cocoa-92103_kubernetes -contexts: -- context: - cluster: lithe-cocoa-92103_kubernetes - user: lithe-cocoa-92103_kubernetes - name: lithe-cocoa-92103_kubernetes -current-context: lithe-cocoa-92103_kubernetes -kind: Config -preferences: {} -users: -- name: lithe-cocoa-92103_kubernetes - user: - client-certificate-data: REDACTED - client-key-data: REDACTED - token: 65rZW78y8HbwXXtSXuUw9DbP4FLjHi4b -- name: lithe-cocoa-92103_kubernetes-basic-auth - user: - password: h5M0FtUUIflBSdI7 - username: admin -{% endraw %} -{% endhighlight %} - -The next step is to define a context for the kubectl client to work in each namespace. The value of "cluster" and "user" fields are copied from the current context. - -{% highlight console %} -{% raw %} -$ kubectl config set-context dev --namespace=development --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes -$ kubectl config set-context prod --namespace=production --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes -{% endraw %} -{% endhighlight %} - -The above commands provided two request contexts you can alternate against depending on what namespace you -wish to work against. - -Let's switch to operate in the development namespace. - -{% highlight console %} -{% raw %} -$ kubectl config use-context dev -{% endraw %} -{% endhighlight %} - -You can verify your current context by doing the following: - -{% highlight console %} -{% raw %} -$ kubectl config view -{% endraw %} -{% endhighlight %} - -{% highlight yaml %} -{% raw %} -apiVersion: v1 -clusters: -- cluster: - certificate-authority-data: REDACTED - server: https://130.211.122.180 - name: lithe-cocoa-92103_kubernetes -contexts: -- context: - cluster: lithe-cocoa-92103_kubernetes - namespace: development - user: lithe-cocoa-92103_kubernetes - name: dev -- context: - cluster: lithe-cocoa-92103_kubernetes - user: lithe-cocoa-92103_kubernetes - name: lithe-cocoa-92103_kubernetes -- context: - cluster: lithe-cocoa-92103_kubernetes - namespace: production - user: lithe-cocoa-92103_kubernetes - name: prod -current-context: dev -kind: Config -preferences: {} -users: -- name: lithe-cocoa-92103_kubernetes - user: - client-certificate-data: REDACTED - client-key-data: REDACTED - token: 65rZW78y8HbwXXtSXuUw9DbP4FLjHi4b -- name: lithe-cocoa-92103_kubernetes-basic-auth - user: - password: h5M0FtUUIflBSdI7 - username: admin -{% endraw %} -{% endhighlight %} - -At this point, all requests we make to the Kubernetes cluster from the command line are scoped to the development namespace. - -Let's create some content. - -{% highlight console %} -{% raw %} -$ kubectl run snowflake --image=kubernetes/serve_hostname --replicas=2 -{% endraw %} -{% endhighlight %} - -We have just created a replication controller whose replica size is 2 that is running the pod called snowflake with a basic container that just serves the hostname. - -{% highlight console %} -{% raw %} -$ kubectl get rc -CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -snowflake snowflake kubernetes/serve_hostname run=snowflake 2 - -$ kubectl get pods -NAME READY STATUS RESTARTS AGE -snowflake-8w0qn 1/1 Running 0 22s -snowflake-jrpzb 1/1 Running 0 22s -{% endraw %} -{% endhighlight %} - -And this is great, developers are able to do what they want, and they do not have to worry about affecting content in the production namespace. - -Let's switch to the production namespace and show how resources in one namespace are hidden from the other. - -{% highlight console %} -{% raw %} -$ kubectl config use-context prod -{% endraw %} -{% endhighlight %} - -The production namespace should be empty. - -{% highlight console %} -{% raw %} -$ kubectl get rc -CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS - -$ kubectl get pods -NAME READY STATUS RESTARTS AGE -{% endraw %} -{% endhighlight %} - -Production likes to run cattle, so let's create some cattle pods. - -{% highlight console %} -{% raw %} -$ kubectl run cattle --image=kubernetes/serve_hostname --replicas=5 - -$ kubectl get rc -CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -cattle cattle kubernetes/serve_hostname run=cattle 5 - -$ kubectl get pods -NAME READY STATUS RESTARTS AGE -cattle-97rva 1/1 Running 0 12s -cattle-i9ojn 1/1 Running 0 12s -cattle-qj3yv 1/1 Running 0 12s -cattle-yc7vn 1/1 Running 0 12s -cattle-zz7ea 1/1 Running 0 12s -{% endraw %} -{% endhighlight %} - -At this point, it should be clear that the resources users create in one namespace are hidden from the other namespace. - -As the policy support in Kubernetes evolves, we will extend this scenario to show how you can provide different +--- +title: "Kubernetes Namespaces" +--- + +Kubernetes _[namespaces](/{{page.version}}/docs/admin/namespaces)_ help different projects, teams, or customers to share a Kubernetes cluster. + +It does this by providing the following: + +1. A scope for [Names](../../user-guide/identifiers). +2. A mechanism to attach authorization and policy to a subsection of the cluster. + +Use of multiple namespaces is optional. + +This example demonstrates how to use Kubernetes namespaces to subdivide your cluster. + +### Step Zero: Prerequisites + +This example assumes the following: + +1. You have an [existing Kubernetes cluster](../../getting-started-guides/). +2. You have a basic understanding of Kubernetes _[pods](../../user-guide/pods)_, _[services](../../user-guide/services)_, and _[replication controllers](../../user-guide/replication-controller)_. + +### Step One: Understand the default namespace + +By default, a Kubernetes cluster will instantiate a default namespace when provisioning the cluster to hold the default set of pods, +services, and replication controllers used by the cluster. + +Assuming you have a fresh cluster, you can introspect the available namespace's by doing the following: + +{% highlight console %} +$ kubectl get namespaces +NAME LABELS +default +{% endhighlight %} + +### Step Two: Create new namespaces + +For this exercise, we will create two additional Kubernetes namespaces to hold our content. + +Let's imagine a scenario where an organization is using a shared Kubernetes cluster for development and production use cases. + +The development team would like to maintain a space in the cluster where they can get a view on the list of pods, services, and replication controllers +they use to build and run their application. In this space, Kubernetes resources come and go, and the restrictions on who can or cannot modify resources +are relaxed to enable agile development. + +The operations team would like to maintain a space in the cluster where they can enforce strict procedures on who can or cannot manipulate the set of +pods, services, and replication controllers that run the production site. + +One pattern this organization could follow is to partition the Kubernetes cluster into two namespaces: development and production. + +Let's create two new namespaces to hold our work. + +Use the file [`namespace-dev.json`](namespace-dev.json) which describes a development namespace: + + + +{% highlight json %} +{ + "kind": "Namespace", + "apiVersion": "v1", + "metadata": { + "name": "development", + "labels": { + "name": "development" + } + } +} +{% endhighlight %} + +[Download example](namespace-dev.json) + + +Create the development namespace using kubectl. + +{% highlight console %} +$ kubectl create -f docs/admin/namespaces/namespace-dev.json +{% endhighlight %} + +And then lets create the production namespace using kubectl. + +{% highlight console %} +$ kubectl create -f docs/admin/namespaces/namespace-prod.json +{% endhighlight %} + +To be sure things are right, let's list all of the namespaces in our cluster. + +{% highlight console %} +$ kubectl get namespaces +NAME LABELS STATUS +default Active +development name=development Active +production name=production Active +{% endhighlight %} + + +### Step Three: Create pods in each namespace + +A Kubernetes namespace provides the scope for pods, services, and replication controllers in the cluster. + +Users interacting with one namespace do not see the content in another namespace. + +To demonstrate this, let's spin up a simple replication controller and pod in the development namespace. + +We first check what is the current context: + +{% highlight yaml %} +apiVersion: v1 +clusters: +- cluster: + certificate-authority-data: REDACTED + server: https://130.211.122.180 + name: lithe-cocoa-92103_kubernetes +contexts: +- context: + cluster: lithe-cocoa-92103_kubernetes + user: lithe-cocoa-92103_kubernetes + name: lithe-cocoa-92103_kubernetes +current-context: lithe-cocoa-92103_kubernetes +kind: Config +preferences: {} +users: +- name: lithe-cocoa-92103_kubernetes + user: + client-certificate-data: REDACTED + client-key-data: REDACTED + token: 65rZW78y8HbwXXtSXuUw9DbP4FLjHi4b +- name: lithe-cocoa-92103_kubernetes-basic-auth + user: + password: h5M0FtUUIflBSdI7 + username: admin +{% endhighlight %} + +The next step is to define a context for the kubectl client to work in each namespace. The value of "cluster" and "user" fields are copied from the current context. + +{% highlight console %} +$ kubectl config set-context dev --namespace=development --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes +$ kubectl config set-context prod --namespace=production --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes +{% endhighlight %} + +The above commands provided two request contexts you can alternate against depending on what namespace you +wish to work against. + +Let's switch to operate in the development namespace. + +{% highlight console %} +$ kubectl config use-context dev +{% endhighlight %} + +You can verify your current context by doing the following: + +{% highlight console %} +$ kubectl config view +{% endhighlight %} + +{% highlight yaml %} +apiVersion: v1 +clusters: +- cluster: + certificate-authority-data: REDACTED + server: https://130.211.122.180 + name: lithe-cocoa-92103_kubernetes +contexts: +- context: + cluster: lithe-cocoa-92103_kubernetes + namespace: development + user: lithe-cocoa-92103_kubernetes + name: dev +- context: + cluster: lithe-cocoa-92103_kubernetes + user: lithe-cocoa-92103_kubernetes + name: lithe-cocoa-92103_kubernetes +- context: + cluster: lithe-cocoa-92103_kubernetes + namespace: production + user: lithe-cocoa-92103_kubernetes + name: prod +current-context: dev +kind: Config +preferences: {} +users: +- name: lithe-cocoa-92103_kubernetes + user: + client-certificate-data: REDACTED + client-key-data: REDACTED + token: 65rZW78y8HbwXXtSXuUw9DbP4FLjHi4b +- name: lithe-cocoa-92103_kubernetes-basic-auth + user: + password: h5M0FtUUIflBSdI7 + username: admin +{% endhighlight %} + +At this point, all requests we make to the Kubernetes cluster from the command line are scoped to the development namespace. + +Let's create some content. + +{% highlight console %} +$ kubectl run snowflake --image=kubernetes/serve_hostname --replicas=2 +{% endhighlight %} + +We have just created a replication controller whose replica size is 2 that is running the pod called snowflake with a basic container that just serves the hostname. + +{% highlight console %} +$ kubectl get rc +CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS +snowflake snowflake kubernetes/serve_hostname run=snowflake 2 + +$ kubectl get pods +NAME READY STATUS RESTARTS AGE +snowflake-8w0qn 1/1 Running 0 22s +snowflake-jrpzb 1/1 Running 0 22s +{% endhighlight %} + +And this is great, developers are able to do what they want, and they do not have to worry about affecting content in the production namespace. + +Let's switch to the production namespace and show how resources in one namespace are hidden from the other. + +{% highlight console %} +$ kubectl config use-context prod +{% endhighlight %} + +The production namespace should be empty. + +{% highlight console %} +$ kubectl get rc +CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS + +$ kubectl get pods +NAME READY STATUS RESTARTS AGE +{% endhighlight %} + +Production likes to run cattle, so let's create some cattle pods. + +{% highlight console %} +$ kubectl run cattle --image=kubernetes/serve_hostname --replicas=5 + +$ kubectl get rc +CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS +cattle cattle kubernetes/serve_hostname run=cattle 5 + +$ kubectl get pods +NAME READY STATUS RESTARTS AGE +cattle-97rva 1/1 Running 0 12s +cattle-i9ojn 1/1 Running 0 12s +cattle-qj3yv 1/1 Running 0 12s +cattle-yc7vn 1/1 Running 0 12s +cattle-zz7ea 1/1 Running 0 12s +{% endhighlight %} + +At this point, it should be clear that the resources users create in one namespace are hidden from the other namespace. + +As the policy support in Kubernetes evolves, we will extend this scenario to show how you can provide different authorization rules for each namespace. \ No newline at end of file diff --git a/v1.1/docs/admin/namespaces/index.md b/v1.1/docs/admin/namespaces/index.md index 321e81714e..849b666ea6 100644 --- a/v1.1/docs/admin/namespaces/index.md +++ b/v1.1/docs/admin/namespaces/index.md @@ -1,15 +1,11 @@ --- title: "Kubernetes Namespaces" --- - - -## Kubernetes Namespaces - -Kubernetes _[namespaces](../../../docs/admin/namespaces.html)_ help different projects, teams, or customers to share a Kubernetes cluster. +Kubernetes _[namespaces](/{{page.version}}/docs/admin/namespaces)_ help different projects, teams, or customers to share a Kubernetes cluster. It does this by providing the following: -1. A scope for [Names](../../user-guide/identifiers.html). +1. A scope for [Names](../../user-guide/identifiers). 2. A mechanism to attach authorization and policy to a subsection of the cluster. Use of multiple namespaces is optional. @@ -21,7 +17,7 @@ This example demonstrates how to use Kubernetes namespaces to subdivide your clu This example assumes the following: 1. You have an [existing Kubernetes cluster](../../getting-started-guides/). -2. You have a basic understanding of Kubernetes _[pods](../../user-guide/pods.html)_, _[services](../../user-guide/services.html)_, and _[replication controllers](../../user-guide/replication-controller.html)_. +2. You have a basic understanding of Kubernetes _[pods](../../user-guide/pods)_, _[services](../../user-guide/services)_, and _[replication controllers](../../user-guide/replication-controller)_. ### Step One: Understand the default namespace @@ -31,11 +27,11 @@ services, and replication controllers used by the cluster. Assuming you have a fresh cluster, you can introspect the available namespace's by doing the following: {% highlight console %} -{% raw %} + $ kubectl get namespaces NAME LABELS default -{% endraw %} + {% endhighlight %} ### Step Two: Create new namespaces @@ -60,7 +56,7 @@ Use the file [`namespace-dev.json`](namespace-dev.json) which describes a develo {% highlight json %} -{% raw %} + { "kind": "Namespace", "apiVersion": "v1", @@ -71,7 +67,7 @@ Use the file [`namespace-dev.json`](namespace-dev.json) which describes a develo } } } -{% endraw %} + {% endhighlight %} [Download example](namespace-dev.json) @@ -80,29 +76,29 @@ Use the file [`namespace-dev.json`](namespace-dev.json) which describes a develo Create the development namespace using kubectl. {% highlight console %} -{% raw %} + $ kubectl create -f docs/admin/namespaces/namespace-dev.json -{% endraw %} + {% endhighlight %} And then lets create the production namespace using kubectl. {% highlight console %} -{% raw %} + $ kubectl create -f docs/admin/namespaces/namespace-prod.json -{% endraw %} + {% endhighlight %} To be sure things are right, let's list all of the namespaces in our cluster. {% highlight console %} -{% raw %} + $ kubectl get namespaces NAME LABELS STATUS default Active development name=development Active production name=production Active -{% endraw %} + {% endhighlight %} @@ -117,7 +113,7 @@ To demonstrate this, let's spin up a simple replication controller and pod in th We first check what is the current context: {% highlight yaml %} -{% raw %} + apiVersion: v1 clusters: - cluster: @@ -142,16 +138,16 @@ users: user: password: h5M0FtUUIflBSdI7 username: admin -{% endraw %} + {% endhighlight %} The next step is to define a context for the kubectl client to work in each namespace. The value of "cluster" and "user" fields are copied from the current context. {% highlight console %} -{% raw %} + $ kubectl config set-context dev --namespace=development --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes $ kubectl config set-context prod --namespace=production --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes -{% endraw %} + {% endhighlight %} The above commands provided two request contexts you can alternate against depending on what namespace you @@ -160,21 +156,21 @@ wish to work against. Let's switch to operate in the development namespace. {% highlight console %} -{% raw %} + $ kubectl config use-context dev -{% endraw %} + {% endhighlight %} You can verify your current context by doing the following: {% highlight console %} -{% raw %} + $ kubectl config view -{% endraw %} + {% endhighlight %} {% highlight yaml %} -{% raw %} + apiVersion: v1 clusters: - cluster: @@ -209,7 +205,7 @@ users: user: password: h5M0FtUUIflBSdI7 username: admin -{% endraw %} + {% endhighlight %} At this point, all requests we make to the Kubernetes cluster from the command line are scoped to the development namespace. @@ -217,15 +213,15 @@ At this point, all requests we make to the Kubernetes cluster from the command l Let's create some content. {% highlight console %} -{% raw %} + $ kubectl run snowflake --image=kubernetes/serve_hostname --replicas=2 -{% endraw %} + {% endhighlight %} We have just created a replication controller whose replica size is 2 that is running the pod called snowflake with a basic container that just serves the hostname. {% highlight console %} -{% raw %} + $ kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS snowflake snowflake kubernetes/serve_hostname run=snowflake 2 @@ -234,7 +230,7 @@ $ kubectl get pods NAME READY STATUS RESTARTS AGE snowflake-8w0qn 1/1 Running 0 22s snowflake-jrpzb 1/1 Running 0 22s -{% endraw %} + {% endhighlight %} And this is great, developers are able to do what they want, and they do not have to worry about affecting content in the production namespace. @@ -242,27 +238,27 @@ And this is great, developers are able to do what they want, and they do not hav Let's switch to the production namespace and show how resources in one namespace are hidden from the other. {% highlight console %} -{% raw %} + $ kubectl config use-context prod -{% endraw %} + {% endhighlight %} The production namespace should be empty. {% highlight console %} -{% raw %} + $ kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS $ kubectl get pods NAME READY STATUS RESTARTS AGE -{% endraw %} + {% endhighlight %} Production likes to run cattle, so let's create some cattle pods. {% highlight console %} -{% raw %} + $ kubectl run cattle --image=kubernetes/serve_hostname --replicas=5 $ kubectl get rc @@ -276,7 +272,7 @@ cattle-i9ojn 1/1 Running 0 12s cattle-qj3yv 1/1 Running 0 12s cattle-yc7vn 1/1 Running 0 12s cattle-zz7ea 1/1 Running 0 12s -{% endraw %} + {% endhighlight %} At this point, it should be clear that the resources users create in one namespace are hidden from the other namespace. diff --git a/v1.1/docs/admin/networking.md b/v1.1/docs/admin/networking.md index 429053591c..be6ce972c7 100644 --- a/v1.1/docs/admin/networking.md +++ b/v1.1/docs/admin/networking.md @@ -1,35 +1,15 @@ --- title: "Networking in Kubernetes" --- - - -# Networking in Kubernetes - -**Table of Contents** - - -- [Networking in Kubernetes](#networking-in-kubernetes) - - [Summary](#summary) - - [Docker model](#docker-model) - - [Kubernetes model](#kubernetes-model) - - [How to achieve this](#how-to-achieve-this) - - [Google Compute Engine (GCE)](#google-compute-engine-gce) - - [L2 networks and linux bridging](#l2-networks-and-linux-bridging) - - [Flannel](#flannel) - - [OpenVSwitch](#openvswitch) - - [Weave](#weave) - - [Calico](#calico) - - [Other reading](#other-reading) - - +{% include pagetoc.html %} Kubernetes approaches networking somewhat differently than Docker does by default. There are 4 distinct networking problems to solve: 1. Highly-coupled container-to-container communications: this is solved by - [pods](../user-guide/pods.html) and `localhost` communications. + [pods](../user-guide/pods) and `localhost` communications. 2. Pod-to-Pod communications: this is the primary focus of this document. -3. Pod-to-Service communications: this is covered by [services](../user-guide/services.html). -4. External-to-Service communications: this is covered by [services](../user-guide/services.html). +3. Pod-to-Service communications: this is covered by [services](../user-guide/services). +4. External-to-Service communications: this is covered by [services](../user-guide/services). ## Summary @@ -95,7 +75,7 @@ talk to other VMs in your project. This is the same basic model. Until now this document has talked about containers. In reality, Kubernetes applies IP addresses at the `Pod` scope - containers within a `Pod` share their network namespaces - including their IP address. This means that containers -within a `Pod` can all reach each other’s ports on `localhost`. This does imply +within a `Pod` can all reach each other's ports on `localhost`. This does imply that containers within a `Pod` must coordinate port usage, but this is no different than processes in a VM. We call this the "IP-per-pod" model. This is implemented in Docker as a "pod container" which holds the network namespace @@ -128,9 +108,9 @@ on that subnet, and is passed to docker's `--bridge` flag. We start Docker with: {% highlight sh %} -{% raw %} + DOCKER_OPTS="--bridge=cbr0 --iptables=false --ip-masq=false" -{% endraw %} + {% endhighlight %} This bridge is created by Kubelet (controlled by the `--configure-cbr0=true` @@ -147,18 +127,18 @@ itself) traffic that is bound for IPs outside the GCE project network (10.0.0.0/8). {% highlight sh %} -{% raw %} + iptables -t nat -A POSTROUTING ! -d 10.0.0.0/8 -o eth0 -j MASQUERADE -{% endraw %} + {% endhighlight %} Lastly we enable IP forwarding in the kernel (so the kernel will process packets for bridged containers): {% highlight sh %} -{% raw %} + sysctl net.ipv4.ip_forward=1 -{% endraw %} + {% endhighlight %} The result of all this is that all `Pods` can reach each other and can egress @@ -185,7 +165,7 @@ people have reported success with Flannel and Kubernetes. ### OpenVSwitch -[OpenVSwitch](ovs-networking.html) is a somewhat more mature but also +[OpenVSwitch](ovs-networking) is a somewhat more mature but also complicated way to build an overlay network. This is endorsed by several of the "Big Shops" for networking. @@ -203,7 +183,7 @@ IPs. The early design of the networking model and its rationale, and some future plans are described in more detail in the [networking design -document](../design/networking.html). +document](../design/networking). diff --git a/v1.1/docs/admin/node.md b/v1.1/docs/admin/node.md index 14b1beaf1a..ac1420b879 100644 --- a/v1.1/docs/admin/node.md +++ b/v1.1/docs/admin/node.md @@ -1,35 +1,13 @@ --- title: "Node" --- - - -# Node - -**Table of Contents** - - -- [Node](#node) - - [What is a node?](#what-is-a-node) - - [Node Status](#node-status) - - [Node Addresses](#node-addresses) - - [Node Phase](#node-phase) - - [Node Condition](#node-condition) - - [Node Capacity](#node-capacity) - - [Node Info](#node-info) - - [Node Management](#node-management) - - [Node Controller](#node-controller) - - [Self-Registration of Nodes](#self-registration-of-nodes) - - [Manual Node Administration](#manual-node-administration) - - [Node capacity](#node-capacity) - - [API Object](#api-object) - - +{% include pagetoc.html %} ## What is a node? `Node` is a worker machine in Kubernetes, previously known as `Minion`. Node may be a VM or physical machine, depending on the cluster. Each node has -the services necessary to run [Pods](../user-guide/pods.html) and is managed by the master +the services necessary to run [Pods](../user-guide/pods) and is managed by the master components. The services on a node include docker, kubelet and network proxy. See [The Kubernetes Node](../design/architecture.html#the-kubernetes-node) section in the architecture design doc for more details. @@ -79,14 +57,14 @@ Node condition is represented as a json object. For example, the following conditions mean the node is in sane state: {% highlight json %} -{% raw %} + "conditions": [ { "kind": "Ready", "status": "True", }, ] -{% endraw %} + {% endhighlight %} If the Status of the Ready condition @@ -105,7 +83,7 @@ The information is gathered by Kubelet from the node. ## Node Management -Unlike [Pods](../user-guide/pods.html) and [Services](../user-guide/services.html), a Node is not inherently +Unlike [Pods](../user-guide/pods) and [Services](../user-guide/services), a Node is not inherently created by Kubernetes: it is either taken from cloud providers like Google Compute Engine, or from your pool of physical or virtual machines. What this means is that when Kubernetes creates a node, it is really just creating an object that represents the node in its internal state. @@ -113,7 +91,7 @@ After creation, Kubernetes will check whether the node is valid or not. For example, if you try to create a node from the following content: {% highlight json %} -{% raw %} + { "kind": "Node", "apiVersion": "v1", @@ -124,7 +102,7 @@ For example, if you try to create a node from the following content: } } } -{% endraw %} + {% endhighlight %} Kubernetes will create a Node object internally (the representation), and @@ -187,9 +165,9 @@ preparatory step before a node reboot, etc. For example, to mark a node unschedulable, run this command: {% highlight sh %} -{% raw %} + kubectl replace nodes 10.1.2.3 --patch='{"apiVersion": "v1", "unschedulable": true}' -{% endraw %} + {% endhighlight %} Note that pods which are created by a daemonSet controller bypass the Kubernetes scheduler, @@ -212,7 +190,7 @@ If you want to explicitly reserve resources for non-Pod processes, you can creat pod. Use the following template: {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: @@ -225,7 +203,7 @@ spec: limits: cpu: 100m memory: 100Mi -{% endraw %} + {% endhighlight %} Set the `cpu` and `memory` values to the amount of resources you want to reserve. diff --git a/v1.1/docs/admin/ovs-networking.md b/v1.1/docs/admin/ovs-networking.md index 37d11e2c4c..07c5e626d8 100644 --- a/v1.1/docs/admin/ovs-networking.md +++ b/v1.1/docs/admin/ovs-networking.md @@ -1,10 +1,6 @@ --- title: "Kubernetes OpenVSwitch GRE/VxLAN networking" --- - - -# Kubernetes OpenVSwitch GRE/VxLAN networking - This document describes how OpenVSwitch is used to setup networking between pods across nodes. The tunnel type could be GRE or VxLAN. VxLAN is preferable when large scale isolation needs to be performed within the network. diff --git a/v1.1/docs/admin/resource-quota.md b/v1.1/docs/admin/resource-quota.md index 67c297d245..9e0c22adb1 100755 --- a/v1.1/docs/admin/resource-quota.md +++ b/v1.1/docs/admin/resource-quota.md @@ -1,10 +1,6 @@ --- title: "Resource Quotas" --- - - -# Resource Quotas - When several users or teams share a cluster with a fixed number of nodes, there is a concern that one team could use more than its fair share of resources. @@ -21,7 +17,7 @@ work like this: their resource requests defaulted to match their defined limits. The user is only charged for the resources they request in the Resource Quota versus their limits because the request is the minimum amount of resource guaranteed by the cluster during scheduling. For more information on over commit, - see [compute-resources](../user-guide/compute-resources.html). + see [compute-resources](../user-guide/compute-resources). - If creating a pod would cause the namespace to exceed any of the limits specified in the the Resource Quota for that namespace, then the request will fail with HTTP status code `403 FORBIDDEN`. @@ -29,7 +25,7 @@ work like this: of the resources for which quota is enabled, then the POST of the pod will fail with HTTP status code `403 FORBIDDEN`. Hint: Use the LimitRange admission controller to force default values of *limits* (then resource *requests* would be equal to *limits* by default, see - [admission controller](admission-controllers.html)) before the quota is checked to avoid this problem. + [admission controller](admission-controllers)) before the quota is checked to avoid this problem. Examples of policies that could be created using namespaces and quotas are: - In a cluster with a capacity of 32 GiB RAM, and 16 cores, let team A use 20 Gib and 10 cores, @@ -54,7 +50,7 @@ Resource Quota is enforced in a particular namespace when there is a ## Compute Resource Quota -The total sum of [compute resources](../user-guide/compute-resources.html) requested by pods +The total sum of [compute resources](../user-guide/compute-resources) requested by pods in a namespace can be limited. The following compute resource types are supported: | ResourceName | Description | @@ -91,7 +87,7 @@ supply of Pod IPs. Kubectl supports creating, updating, and viewing quotas: {% highlight console %} -{% raw %} + $ kubectl namespace myspace $ cat < quota.json { @@ -126,7 +122,7 @@ pods 5 10 replicationcontrollers 5 20 resourcequotas 1 1 services 3 5 -{% endraw %} + {% endhighlight %} ## Quota and Cluster Capacity @@ -154,7 +150,7 @@ See a [detailed example for how to use resource quota](resourcequota/).. ## Read More -See [ResourceQuota design doc](../design/admission_control_resource_quota.html) for more information. +See [ResourceQuota design doc](../design/admission_control_resource_quota) for more information. diff --git a/v1.1/docs/admin/resourcequota/README.md b/v1.1/docs/admin/resourcequota/README.md index 0527b3191e..047a5901ad 100755 --- a/v1.1/docs/admin/resourcequota/README.md +++ b/v1.1/docs/admin/resourcequota/README.md @@ -1,183 +1,165 @@ ---- -title: "Resource Quota" ---- - -Resource Quota -======================================== -This example demonstrates how [resource quota](../../admin/admission-controllers.html#resourcequota) and -[limitsranger](../../admin/admission-controllers.html#limitranger) can be applied to a Kubernetes namespace. -See [ResourceQuota design doc](../../design/admission_control_resource_quota.html) for more information. - -This example assumes you have a functional Kubernetes setup. - -Step 1: Create a namespace ------------------------------------------ -This example will work in a custom namespace to demonstrate the concepts involved. - -Let's create a new namespace called quota-example: - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/resourcequota/namespace.yaml -namespace "quota-example" created -$ kubectl get namespaces -NAME LABELS STATUS AGE -default Active 2m -quota-example Active 39s -{% endraw %} -{% endhighlight %} - -Step 2: Apply a quota to the namespace ------------------------------------------ -By default, a pod will run with unbounded CPU and memory requests/limits. This means that any pod in the -system will be able to consume as much CPU and memory on the node that executes the pod. - -Users may want to restrict how much of the cluster resources a given namespace may consume -across all of its pods in order to manage cluster usage. To do this, a user applies a quota to -a namespace. A quota lets the user set hard limits on the total amount of node resources (cpu, memory) -and API resources (pods, services, etc.) that a namespace may consume. In term of resources, Kubernetes -checks the total resource *requests*, not resource *limits* of all containers/pods in the namespace. - -Let's create a simple quota in our namespace: - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/resourcequota/quota.yaml --namespace=quota-example -resourcequota "quota" created -{% endraw %} -{% endhighlight %} - -Once your quota is applied to a namespace, the system will restrict any creation of content -in the namespace until the quota usage has been calculated. This should happen quickly. - -You can describe your current quota usage to see what resources are being consumed in your -namespace. - -{% highlight console %} -{% raw %} -$ kubectl describe quota quota --namespace=quota-example -Name: quota -Namespace: quota-example -Resource Used Hard --------- ---- ---- -cpu 0 20 -memory 0 1Gi -persistentvolumeclaims 0 10 -pods 0 10 -replicationcontrollers 0 20 -resourcequotas 1 1 -secrets 1 10 -services 0 5 -{% endraw %} -{% endhighlight %} - -Step 3: Applying default resource requests and limits ------------------------------------------ -Pod authors rarely specify resource requests and limits for their pods. - -Since we applied a quota to our project, let's see what happens when an end-user creates a pod that has unbounded -cpu and memory by creating an nginx container. - -To demonstrate, lets create a replication controller that runs nginx: - -{% highlight console %} -{% raw %} -$ kubectl run nginx --image=nginx --replicas=1 --namespace=quota-example -replicationcontroller "nginx" created -{% endraw %} -{% endhighlight %} - -Now let's look at the pods that were created. - -{% highlight console %} -{% raw %} -$ kubectl get pods --namespace=quota-example -NAME READY STATUS RESTARTS AGE -{% endraw %} -{% endhighlight %} - -What happened? I have no pods! Let's describe the replication controller to get a view of what is happening. - -{% highlight console %} -{% raw %} -kubectl describe rc nginx --namespace=quota-example -Name: nginx -Namespace: quota-example -Image(s): nginx -Selector: run=nginx -Labels: run=nginx -Replicas: 0 current / 1 desired -Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed -No volumes. -Events: - FirstSeen LastSeen Count From SubobjectPath Reason Message - 42s 11s 3 {replication-controller } FailedCreate Error creating: Pod "nginx-" is forbidden: Must make a non-zero request for memory since it is tracked by quota. -{% endraw %} -{% endhighlight %} - -The Kubernetes API server is rejecting the replication controllers requests to create a pod because our pods -do not specify any memory usage *request*. - -So let's set some default values for the amount of cpu and memory a pod can consume: - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/resourcequota/limits.yaml --namespace=quota-example -limitrange "limits" created -$ kubectl describe limits limits --namespace=quota-example -Name: limits -Namespace: quota-example -Type Resource Min Max Request Limit Limit/Request ----- -------- --- --- ------- ----- ------------- -Container memory - - 256Mi 512Mi - -Container cpu - - 100m 200m - -{% endraw %} -{% endhighlight %} - -Now any time a pod is created in this namespace, if it has not specified any resource request/limit, the default -amount of cpu and memory per container will be applied, and the request will be used as part of admission control. - -Now that we have applied default resource *request* for our namespace, our replication controller should be able to -create its pods. - -{% highlight console %} -{% raw %} -$ kubectl get pods --namespace=quota-example -NAME READY STATUS RESTARTS AGE -nginx-fca65 1/1 Running 0 1m -{% endraw %} -{% endhighlight %} - -And if we print out our quota usage in the namespace: - -{% highlight console %} -{% raw %} -$ kubectl describe quota quota --namespace=quota-example -Name: quota -Namespace: quota-example -Resource Used Hard --------- ---- ---- -cpu 100m 20 -memory 256Mi 1Gi -persistentvolumeclaims 0 10 -pods 1 10 -replicationcontrollers 1 20 -resourcequotas 1 1 -secrets 1 10 -services 0 5 -{% endraw %} -{% endhighlight %} - -You can now see the pod that was created is consuming explicit amounts of resources (specified by resource *request*), -and the usage is being tracked by the Kubernetes system properly. - -Summary ----------------------------- -Actions that consume node resources for cpu and memory can be subject to hard quota limits defined -by the namespace quota. The resource consumption is measured by resource *request* in pod specification. - -Any action that consumes those resources can be tweaked, or can pick up namespace level defaults to -meet your end goal. - - - +--- +title: "Resource Quota" +--- + +Resource Quota +======================================== +This example demonstrates how [resource quota](../../admin/admission-controllers.html#resourcequota) and +[limitsranger](../../admin/admission-controllers.html#limitranger) can be applied to a Kubernetes namespace. +See [ResourceQuota design doc](../../design/admission_control_resource_quota) for more information. + +This example assumes you have a functional Kubernetes setup. + +Step 1: Create a namespace +----------------------------------------- +This example will work in a custom namespace to demonstrate the concepts involved. + +Let's create a new namespace called quota-example: + +{% highlight console %} +$ kubectl create -f docs/admin/resourcequota/namespace.yaml +namespace "quota-example" created +$ kubectl get namespaces +NAME LABELS STATUS AGE +default Active 2m +quota-example Active 39s +{% endhighlight %} + +Step 2: Apply a quota to the namespace +----------------------------------------- +By default, a pod will run with unbounded CPU and memory requests/limits. This means that any pod in the +system will be able to consume as much CPU and memory on the node that executes the pod. + +Users may want to restrict how much of the cluster resources a given namespace may consume +across all of its pods in order to manage cluster usage. To do this, a user applies a quota to +a namespace. A quota lets the user set hard limits on the total amount of node resources (cpu, memory) +and API resources (pods, services, etc.) that a namespace may consume. In term of resources, Kubernetes +checks the total resource *requests*, not resource *limits* of all containers/pods in the namespace. + +Let's create a simple quota in our namespace: + +{% highlight console %} +$ kubectl create -f docs/admin/resourcequota/quota.yaml --namespace=quota-example +resourcequota "quota" created +{% endhighlight %} + +Once your quota is applied to a namespace, the system will restrict any creation of content +in the namespace until the quota usage has been calculated. This should happen quickly. + +You can describe your current quota usage to see what resources are being consumed in your +namespace. + +{% highlight console %} +$ kubectl describe quota quota --namespace=quota-example +Name: quota +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +cpu 0 20 +memory 0 1Gi +persistentvolumeclaims 0 10 +pods 0 10 +replicationcontrollers 0 20 +resourcequotas 1 1 +secrets 1 10 +services 0 5 +{% endhighlight %} + +Step 3: Applying default resource requests and limits +----------------------------------------- +Pod authors rarely specify resource requests and limits for their pods. + +Since we applied a quota to our project, let's see what happens when an end-user creates a pod that has unbounded +cpu and memory by creating an nginx container. + +To demonstrate, lets create a replication controller that runs nginx: + +{% highlight console %} +$ kubectl run nginx --image=nginx --replicas=1 --namespace=quota-example +replicationcontroller "nginx" created +{% endhighlight %} + +Now let's look at the pods that were created. + +{% highlight console %} +$ kubectl get pods --namespace=quota-example +NAME READY STATUS RESTARTS AGE +{% endhighlight %} + +What happened? I have no pods! Let's describe the replication controller to get a view of what is happening. + +{% highlight console %} +kubectl describe rc nginx --namespace=quota-example +Name: nginx +Namespace: quota-example +Image(s): nginx +Selector: run=nginx +Labels: run=nginx +Replicas: 0 current / 1 desired +Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed +No volumes. +Events: + FirstSeen LastSeen Count From SubobjectPath Reason Message + 42s 11s 3 {replication-controller } FailedCreate Error creating: Pod "nginx-" is forbidden: Must make a non-zero request for memory since it is tracked by quota. +{% endhighlight %} + +The Kubernetes API server is rejecting the replication controllers requests to create a pod because our pods +do not specify any memory usage *request*. + +So let's set some default values for the amount of cpu and memory a pod can consume: + +{% highlight console %} +$ kubectl create -f docs/admin/resourcequota/limits.yaml --namespace=quota-example +limitrange "limits" created +$ kubectl describe limits limits --namespace=quota-example +Name: limits +Namespace: quota-example +Type Resource Min Max Request Limit Limit/Request +---- -------- --- --- ------- ----- ------------- +Container memory - - 256Mi 512Mi - +Container cpu - - 100m 200m - +{% endhighlight %} + +Now any time a pod is created in this namespace, if it has not specified any resource request/limit, the default +amount of cpu and memory per container will be applied, and the request will be used as part of admission control. + +Now that we have applied default resource *request* for our namespace, our replication controller should be able to +create its pods. + +{% highlight console %} +$ kubectl get pods --namespace=quota-example +NAME READY STATUS RESTARTS AGE +nginx-fca65 1/1 Running 0 1m +{% endhighlight %} + +And if we print out our quota usage in the namespace: + +{% highlight console %} +$ kubectl describe quota quota --namespace=quota-example +Name: quota +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +cpu 100m 20 +memory 256Mi 1Gi +persistentvolumeclaims 0 10 +pods 1 10 +replicationcontrollers 1 20 +resourcequotas 1 1 +secrets 1 10 +services 0 5 +{% endhighlight %} + +You can now see the pod that was created is consuming explicit amounts of resources (specified by resource *request*), +and the usage is being tracked by the Kubernetes system properly. + +Summary +---------------------------- +Actions that consume node resources for cpu and memory can be subject to hard quota limits defined +by the namespace quota. The resource consumption is measured by resource *request* in pod specification. + +Any action that consumes those resources can be tweaked, or can pick up namespace level defaults to +meet your end goal. + + + diff --git a/v1.1/docs/admin/resourcequota/index.md b/v1.1/docs/admin/resourcequota/index.md index 0527b3191e..047a5901ad 100644 --- a/v1.1/docs/admin/resourcequota/index.md +++ b/v1.1/docs/admin/resourcequota/index.md @@ -1,183 +1,165 @@ ---- -title: "Resource Quota" ---- - -Resource Quota -======================================== -This example demonstrates how [resource quota](../../admin/admission-controllers.html#resourcequota) and -[limitsranger](../../admin/admission-controllers.html#limitranger) can be applied to a Kubernetes namespace. -See [ResourceQuota design doc](../../design/admission_control_resource_quota.html) for more information. - -This example assumes you have a functional Kubernetes setup. - -Step 1: Create a namespace ------------------------------------------ -This example will work in a custom namespace to demonstrate the concepts involved. - -Let's create a new namespace called quota-example: - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/resourcequota/namespace.yaml -namespace "quota-example" created -$ kubectl get namespaces -NAME LABELS STATUS AGE -default Active 2m -quota-example Active 39s -{% endraw %} -{% endhighlight %} - -Step 2: Apply a quota to the namespace ------------------------------------------ -By default, a pod will run with unbounded CPU and memory requests/limits. This means that any pod in the -system will be able to consume as much CPU and memory on the node that executes the pod. - -Users may want to restrict how much of the cluster resources a given namespace may consume -across all of its pods in order to manage cluster usage. To do this, a user applies a quota to -a namespace. A quota lets the user set hard limits on the total amount of node resources (cpu, memory) -and API resources (pods, services, etc.) that a namespace may consume. In term of resources, Kubernetes -checks the total resource *requests*, not resource *limits* of all containers/pods in the namespace. - -Let's create a simple quota in our namespace: - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/resourcequota/quota.yaml --namespace=quota-example -resourcequota "quota" created -{% endraw %} -{% endhighlight %} - -Once your quota is applied to a namespace, the system will restrict any creation of content -in the namespace until the quota usage has been calculated. This should happen quickly. - -You can describe your current quota usage to see what resources are being consumed in your -namespace. - -{% highlight console %} -{% raw %} -$ kubectl describe quota quota --namespace=quota-example -Name: quota -Namespace: quota-example -Resource Used Hard --------- ---- ---- -cpu 0 20 -memory 0 1Gi -persistentvolumeclaims 0 10 -pods 0 10 -replicationcontrollers 0 20 -resourcequotas 1 1 -secrets 1 10 -services 0 5 -{% endraw %} -{% endhighlight %} - -Step 3: Applying default resource requests and limits ------------------------------------------ -Pod authors rarely specify resource requests and limits for their pods. - -Since we applied a quota to our project, let's see what happens when an end-user creates a pod that has unbounded -cpu and memory by creating an nginx container. - -To demonstrate, lets create a replication controller that runs nginx: - -{% highlight console %} -{% raw %} -$ kubectl run nginx --image=nginx --replicas=1 --namespace=quota-example -replicationcontroller "nginx" created -{% endraw %} -{% endhighlight %} - -Now let's look at the pods that were created. - -{% highlight console %} -{% raw %} -$ kubectl get pods --namespace=quota-example -NAME READY STATUS RESTARTS AGE -{% endraw %} -{% endhighlight %} - -What happened? I have no pods! Let's describe the replication controller to get a view of what is happening. - -{% highlight console %} -{% raw %} -kubectl describe rc nginx --namespace=quota-example -Name: nginx -Namespace: quota-example -Image(s): nginx -Selector: run=nginx -Labels: run=nginx -Replicas: 0 current / 1 desired -Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed -No volumes. -Events: - FirstSeen LastSeen Count From SubobjectPath Reason Message - 42s 11s 3 {replication-controller } FailedCreate Error creating: Pod "nginx-" is forbidden: Must make a non-zero request for memory since it is tracked by quota. -{% endraw %} -{% endhighlight %} - -The Kubernetes API server is rejecting the replication controllers requests to create a pod because our pods -do not specify any memory usage *request*. - -So let's set some default values for the amount of cpu and memory a pod can consume: - -{% highlight console %} -{% raw %} -$ kubectl create -f docs/admin/resourcequota/limits.yaml --namespace=quota-example -limitrange "limits" created -$ kubectl describe limits limits --namespace=quota-example -Name: limits -Namespace: quota-example -Type Resource Min Max Request Limit Limit/Request ----- -------- --- --- ------- ----- ------------- -Container memory - - 256Mi 512Mi - -Container cpu - - 100m 200m - -{% endraw %} -{% endhighlight %} - -Now any time a pod is created in this namespace, if it has not specified any resource request/limit, the default -amount of cpu and memory per container will be applied, and the request will be used as part of admission control. - -Now that we have applied default resource *request* for our namespace, our replication controller should be able to -create its pods. - -{% highlight console %} -{% raw %} -$ kubectl get pods --namespace=quota-example -NAME READY STATUS RESTARTS AGE -nginx-fca65 1/1 Running 0 1m -{% endraw %} -{% endhighlight %} - -And if we print out our quota usage in the namespace: - -{% highlight console %} -{% raw %} -$ kubectl describe quota quota --namespace=quota-example -Name: quota -Namespace: quota-example -Resource Used Hard --------- ---- ---- -cpu 100m 20 -memory 256Mi 1Gi -persistentvolumeclaims 0 10 -pods 1 10 -replicationcontrollers 1 20 -resourcequotas 1 1 -secrets 1 10 -services 0 5 -{% endraw %} -{% endhighlight %} - -You can now see the pod that was created is consuming explicit amounts of resources (specified by resource *request*), -and the usage is being tracked by the Kubernetes system properly. - -Summary ----------------------------- -Actions that consume node resources for cpu and memory can be subject to hard quota limits defined -by the namespace quota. The resource consumption is measured by resource *request* in pod specification. - -Any action that consumes those resources can be tweaked, or can pick up namespace level defaults to -meet your end goal. - - - +--- +title: "Resource Quota" +--- + +Resource Quota +======================================== +This example demonstrates how [resource quota](../../admin/admission-controllers.html#resourcequota) and +[limitsranger](../../admin/admission-controllers.html#limitranger) can be applied to a Kubernetes namespace. +See [ResourceQuota design doc](../../design/admission_control_resource_quota) for more information. + +This example assumes you have a functional Kubernetes setup. + +Step 1: Create a namespace +----------------------------------------- +This example will work in a custom namespace to demonstrate the concepts involved. + +Let's create a new namespace called quota-example: + +{% highlight console %} +$ kubectl create -f docs/admin/resourcequota/namespace.yaml +namespace "quota-example" created +$ kubectl get namespaces +NAME LABELS STATUS AGE +default Active 2m +quota-example Active 39s +{% endhighlight %} + +Step 2: Apply a quota to the namespace +----------------------------------------- +By default, a pod will run with unbounded CPU and memory requests/limits. This means that any pod in the +system will be able to consume as much CPU and memory on the node that executes the pod. + +Users may want to restrict how much of the cluster resources a given namespace may consume +across all of its pods in order to manage cluster usage. To do this, a user applies a quota to +a namespace. A quota lets the user set hard limits on the total amount of node resources (cpu, memory) +and API resources (pods, services, etc.) that a namespace may consume. In term of resources, Kubernetes +checks the total resource *requests*, not resource *limits* of all containers/pods in the namespace. + +Let's create a simple quota in our namespace: + +{% highlight console %} +$ kubectl create -f docs/admin/resourcequota/quota.yaml --namespace=quota-example +resourcequota "quota" created +{% endhighlight %} + +Once your quota is applied to a namespace, the system will restrict any creation of content +in the namespace until the quota usage has been calculated. This should happen quickly. + +You can describe your current quota usage to see what resources are being consumed in your +namespace. + +{% highlight console %} +$ kubectl describe quota quota --namespace=quota-example +Name: quota +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +cpu 0 20 +memory 0 1Gi +persistentvolumeclaims 0 10 +pods 0 10 +replicationcontrollers 0 20 +resourcequotas 1 1 +secrets 1 10 +services 0 5 +{% endhighlight %} + +Step 3: Applying default resource requests and limits +----------------------------------------- +Pod authors rarely specify resource requests and limits for their pods. + +Since we applied a quota to our project, let's see what happens when an end-user creates a pod that has unbounded +cpu and memory by creating an nginx container. + +To demonstrate, lets create a replication controller that runs nginx: + +{% highlight console %} +$ kubectl run nginx --image=nginx --replicas=1 --namespace=quota-example +replicationcontroller "nginx" created +{% endhighlight %} + +Now let's look at the pods that were created. + +{% highlight console %} +$ kubectl get pods --namespace=quota-example +NAME READY STATUS RESTARTS AGE +{% endhighlight %} + +What happened? I have no pods! Let's describe the replication controller to get a view of what is happening. + +{% highlight console %} +kubectl describe rc nginx --namespace=quota-example +Name: nginx +Namespace: quota-example +Image(s): nginx +Selector: run=nginx +Labels: run=nginx +Replicas: 0 current / 1 desired +Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed +No volumes. +Events: + FirstSeen LastSeen Count From SubobjectPath Reason Message + 42s 11s 3 {replication-controller } FailedCreate Error creating: Pod "nginx-" is forbidden: Must make a non-zero request for memory since it is tracked by quota. +{% endhighlight %} + +The Kubernetes API server is rejecting the replication controllers requests to create a pod because our pods +do not specify any memory usage *request*. + +So let's set some default values for the amount of cpu and memory a pod can consume: + +{% highlight console %} +$ kubectl create -f docs/admin/resourcequota/limits.yaml --namespace=quota-example +limitrange "limits" created +$ kubectl describe limits limits --namespace=quota-example +Name: limits +Namespace: quota-example +Type Resource Min Max Request Limit Limit/Request +---- -------- --- --- ------- ----- ------------- +Container memory - - 256Mi 512Mi - +Container cpu - - 100m 200m - +{% endhighlight %} + +Now any time a pod is created in this namespace, if it has not specified any resource request/limit, the default +amount of cpu and memory per container will be applied, and the request will be used as part of admission control. + +Now that we have applied default resource *request* for our namespace, our replication controller should be able to +create its pods. + +{% highlight console %} +$ kubectl get pods --namespace=quota-example +NAME READY STATUS RESTARTS AGE +nginx-fca65 1/1 Running 0 1m +{% endhighlight %} + +And if we print out our quota usage in the namespace: + +{% highlight console %} +$ kubectl describe quota quota --namespace=quota-example +Name: quota +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +cpu 100m 20 +memory 256Mi 1Gi +persistentvolumeclaims 0 10 +pods 1 10 +replicationcontrollers 1 20 +resourcequotas 1 1 +secrets 1 10 +services 0 5 +{% endhighlight %} + +You can now see the pod that was created is consuming explicit amounts of resources (specified by resource *request*), +and the usage is being tracked by the Kubernetes system properly. + +Summary +---------------------------- +Actions that consume node resources for cpu and memory can be subject to hard quota limits defined +by the namespace quota. The resource consumption is measured by resource *request* in pod specification. + +Any action that consumes those resources can be tweaked, or can pick up namespace level defaults to +meet your end goal. + + + diff --git a/v1.1/docs/admin/salt.md b/v1.1/docs/admin/salt.md index a3c4edb3ee..b637f61f72 100644 --- a/v1.1/docs/admin/salt.md +++ b/v1.1/docs/admin/salt.md @@ -1,109 +1,100 @@ ---- -title: "Configuring Kubernetes with Salt" -section: guides ---- -The Kubernetes cluster can be configured using Salt. - -The Salt scripts are shared across multiple hosting providers, so it's important to understand some background information prior to making a modification to ensure your changes do not break hosting Kubernetes across multiple environments. Depending on where you host your Kubernetes cluster, you may be using different operating systems and different networking configurations. As a result, it's important to understand some background information before making Salt changes in order to minimize introducing failures for other hosting providers. - -## Salt cluster setup - -The **salt-master** service runs on the kubernetes-master [(except on the default GCE setup)](#standalone-salt-configuration-on-gce). - -The **salt-minion** service runs on the kubernetes-master and each kubernetes-node in the cluster. - -Each salt-minion service is configured to interact with the **salt-master** service hosted on the kubernetes-master via the **master.conf** file [(except on GCE)](#standalone-salt-configuration-on-gce). - -{% highlight console %} -{% raw %} -[root@kubernetes-master] $ cat /etc/salt/minion.d/master.conf -master: kubernetes-master -{% endraw %} -{% endhighlight %} - -The salt-master is contacted by each salt-minion and depending upon the machine information presented, the salt-master will provision the machine as either a kubernetes-master or kubernetes-node with all the required capabilities needed to run Kubernetes. - -If you are running the Vagrant based environment, the **salt-api** service is running on the kubernetes-master. It is configured to enable the vagrant user to introspect the salt cluster in order to find out about machines in the Vagrant environment via a REST API. - -## Standalone Salt Configuration on GCE - -On GCE, the master and nodes are all configured as [standalone minions](http://docs.saltstack.com/en/latest/topics/tutorials/standalone_minion.html). The configuration for each VM is derived from the VM's [instance metadata](https://cloud.google.com/compute/docs/metadata) and then stored in Salt grains (`/etc/salt/minion.d/grains.conf`) and pillars (`/srv/salt-overlay/pillar/cluster-params.sls`) that local Salt uses to enforce state. - -All remaining sections that refer to master/minion setups should be ignored for GCE. One fallout of the GCE setup is that the Salt mine doesn't exist - there is no sharing of configuration amongst nodes. - -## Salt security - -*(Not applicable on default GCE setup.)* - -Security is not enabled on the salt-master, and the salt-master is configured to auto-accept incoming requests from minions. It is not recommended to use this security configuration in production environments without deeper study. (In some environments this isn't as bad as it might sound if the salt master port isn't externally accessible and you trust everyone on your network.) - -{% highlight console %} -{% raw %} -[root@kubernetes-master] $ cat /etc/salt/master.d/auto-accept.conf -open_mode: True -auto_accept: True -{% endraw %} -{% endhighlight %} - -## Salt minion configuration - -Each minion in the salt cluster has an associated configuration that instructs the salt-master how to provision the required resources on the machine. - -An example file is presented below using the Vagrant based environment. - -{% highlight console %} -{% raw %} -[root@kubernetes-master] $ cat /etc/salt/minion.d/grains.conf -grains: - etcd_servers: $MASTER_IP - cloud_provider: vagrant - roles: - - kubernetes-master -{% endraw %} -{% endhighlight %} - -Each hosting environment has a slightly different grains.conf file that is used to build conditional logic where required in the Salt files. - -The following enumerates the set of defined key/value pairs that are supported today. If you add new ones, please make sure to update this list. - -Key | Value -------------- | ------------- -`api_servers` | (Optional) The IP address / host name where a kubelet can get read-only access to kube-apiserver -`cbr-cidr` | (Optional) The minion IP address range used for the docker container bridge. -`cloud` | (Optional) Which IaaS platform is used to host Kubernetes, *gce*, *azure*, *aws*, *vagrant* -`etcd_servers` | (Optional) Comma-delimited list of IP addresses the kube-apiserver and kubelet use to reach etcd. Uses the IP of the first machine in the kubernetes_master role, or 127.0.0.1 on GCE. -`hostnamef` | (Optional) The full host name of the machine, i.e. uname -n -`node_ip` | (Optional) The IP address to use to address this node -`hostname_override` | (Optional) Mapped to the kubelet hostname-override -`network_mode` | (Optional) Networking model to use among nodes: *openvswitch* -`networkInterfaceName` | (Optional) Networking interface to use to bind addresses, default value *eth0* -`publicAddressOverride` | (Optional) The IP address the kube-apiserver should use to bind against for external read-only access -`roles` | (Required) 1. `kubernetes-master` means this machine is the master in the Kubernetes cluster. 2. `kubernetes-pool` means this machine is a kubernetes-node. Depending on the role, the Salt scripts will provision different resources on the machine. - -These keys may be leveraged by the Salt sls files to branch behavior. - -In addition, a cluster may be running a Debian based operating system or Red Hat based operating system (Centos, Fedora, RHEL, etc.). As a result, it's important to sometimes distinguish behavior based on operating system using if branches like the following. - -{% highlight jinja %} -{% raw %} -{% if grains['os_family'] == 'RedHat' %} -// something specific to a RedHat environment (Centos, Fedora, RHEL) where you may use yum, systemd, etc. -{% else %} -// something specific to Debian environment (apt-get, initd) -{% endif %} -{% endraw %} -{% endhighlight %} - -## Best Practices - -1. When configuring default arguments for processes, it's best to avoid the use of EnvironmentFiles (Systemd in Red Hat environments) or init.d files (Debian distributions) to hold default values that should be common across operating system environments. This helps keep our Salt template files easy to understand for editors who may not be familiar with the particulars of each distribution. - -## Future enhancements (Networking) - -Per pod IP configuration is provider-specific, so when making networking changes, it's important to sandbox these as all providers may not use the same mechanisms (iptables, openvswitch, etc.) - -We should define a grains.conf key that captures more specifically what network configuration environment is being used to avoid future confusion across providers. - -## Further reading - +--- +title: "Configuring Kubernetes with Salt" +--- +The Kubernetes cluster can be configured using Salt. + +The Salt scripts are shared across multiple hosting providers, so it's important to understand some background information prior to making a modification to ensure your changes do not break hosting Kubernetes across multiple environments. Depending on where you host your Kubernetes cluster, you may be using different operating systems and different networking configurations. As a result, it's important to understand some background information before making Salt changes in order to minimize introducing failures for other hosting providers. + +## Salt cluster setup + +The **salt-master** service runs on the kubernetes-master [(except on the default GCE setup)](#standalone-salt-configuration-on-gce). + +The **salt-minion** service runs on the kubernetes-master and each kubernetes-node in the cluster. + +Each salt-minion service is configured to interact with the **salt-master** service hosted on the kubernetes-master via the **master.conf** file [(except on GCE)](#standalone-salt-configuration-on-gce). + +{% highlight console %} +[root@kubernetes-master] $ cat /etc/salt/minion.d/master.conf +master: kubernetes-master +{% endhighlight %} + +The salt-master is contacted by each salt-minion and depending upon the machine information presented, the salt-master will provision the machine as either a kubernetes-master or kubernetes-node with all the required capabilities needed to run Kubernetes. + +If you are running the Vagrant based environment, the **salt-api** service is running on the kubernetes-master. It is configured to enable the vagrant user to introspect the salt cluster in order to find out about machines in the Vagrant environment via a REST API. + +## Standalone Salt Configuration on GCE + +On GCE, the master and nodes are all configured as [standalone minions](http://docs.saltstack.com/en/latest/topics/tutorials/standalone_minion). The configuration for each VM is derived from the VM's [instance metadata](https://cloud.google.com/compute/docs/metadata) and then stored in Salt grains (`/etc/salt/minion.d/grains.conf`) and pillars (`/srv/salt-overlay/pillar/cluster-params.sls`) that local Salt uses to enforce state. + +All remaining sections that refer to master/minion setups should be ignored for GCE. One fallout of the GCE setup is that the Salt mine doesn't exist - there is no sharing of configuration amongst nodes. + +## Salt security + +*(Not applicable on default GCE setup.)* + +Security is not enabled on the salt-master, and the salt-master is configured to auto-accept incoming requests from minions. It is not recommended to use this security configuration in production environments without deeper study. (In some environments this isn't as bad as it might sound if the salt master port isn't externally accessible and you trust everyone on your network.) + +{% highlight console %} +[root@kubernetes-master] $ cat /etc/salt/master.d/auto-accept.conf +open_mode: True +auto_accept: True +{% endhighlight %} + +## Salt minion configuration + +Each minion in the salt cluster has an associated configuration that instructs the salt-master how to provision the required resources on the machine. + +An example file is presented below using the Vagrant based environment. + +{% highlight console %} +[root@kubernetes-master] $ cat /etc/salt/minion.d/grains.conf +grains: + etcd_servers: $MASTER_IP + cloud_provider: vagrant + roles: + - kubernetes-master +{% endhighlight %} + +Each hosting environment has a slightly different grains.conf file that is used to build conditional logic where required in the Salt files. + +The following enumerates the set of defined key/value pairs that are supported today. If you add new ones, please make sure to update this list. + +Key | Value +------------- | ------------- +`api_servers` | (Optional) The IP address / host name where a kubelet can get read-only access to kube-apiserver +`cbr-cidr` | (Optional) The minion IP address range used for the docker container bridge. +`cloud` | (Optional) Which IaaS platform is used to host Kubernetes, *gce*, *azure*, *aws*, *vagrant* +`etcd_servers` | (Optional) Comma-delimited list of IP addresses the kube-apiserver and kubelet use to reach etcd. Uses the IP of the first machine in the kubernetes_master role, or 127.0.0.1 on GCE. +`hostnamef` | (Optional) The full host name of the machine, i.e. uname -n +`node_ip` | (Optional) The IP address to use to address this node +`hostname_override` | (Optional) Mapped to the kubelet hostname-override +`network_mode` | (Optional) Networking model to use among nodes: *openvswitch* +`networkInterfaceName` | (Optional) Networking interface to use to bind addresses, default value *eth0* +`publicAddressOverride` | (Optional) The IP address the kube-apiserver should use to bind against for external read-only access +`roles` | (Required) 1. `kubernetes-master` means this machine is the master in the Kubernetes cluster. 2. `kubernetes-pool` means this machine is a kubernetes-node. Depending on the role, the Salt scripts will provision different resources on the machine. + +These keys may be leveraged by the Salt sls files to branch behavior. + +In addition, a cluster may be running a Debian based operating system or Red Hat based operating system (Centos, Fedora, RHEL, etc.). As a result, it's important to sometimes distinguish behavior based on operating system using if branches like the following. + +{% highlight jinja %} +{% if grains['os_family'] == 'RedHat' %} +// something specific to a RedHat environment (Centos, Fedora, RHEL) where you may use yum, systemd, etc. +{% else %} +// something specific to Debian environment (apt-get, initd) +{% endif %} +{% endhighlight %} + +## Best Practices + +1. When configuring default arguments for processes, it's best to avoid the use of EnvironmentFiles (Systemd in Red Hat environments) or init.d files (Debian distributions) to hold default values that should be common across operating system environments. This helps keep our Salt template files easy to understand for editors who may not be familiar with the particulars of each distribution. + +## Future enhancements (Networking) + +Per pod IP configuration is provider-specific, so when making networking changes, it's important to sandbox these as all providers may not use the same mechanisms (iptables, openvswitch, etc.) + +We should define a grains.conf key that captures more specifically what network configuration environment is being used to avoid future confusion across providers. + +## Further reading + The [cluster/saltbase](http://releases.k8s.io/release-1.1/cluster/saltbase/) tree has more details on the current SaltStack configuration. \ No newline at end of file diff --git a/v1.1/docs/admin/service-accounts-admin.md b/v1.1/docs/admin/service-accounts-admin.md index be7c8735fd..24c4ff7bff 100644 --- a/v1.1/docs/admin/service-accounts-admin.md +++ b/v1.1/docs/admin/service-accounts-admin.md @@ -1,12 +1,8 @@ --- title: "Cluster Admin Guide to Service Accounts" --- - - -# Cluster Admin Guide to Service Accounts - *This is a Cluster Administrator guide to service accounts. It assumes knowledge of -the [User Guide to Service Accounts](../user-guide/service-accounts.html).* +the [User Guide to Service Accounts](../user-guide/service-accounts).* *Support for authorization and user accounts is planned but incomplete. Sometimes incomplete features are referred to in order to better describe service accounts.* @@ -40,7 +36,7 @@ Three separate components cooperate to implement the automation around service a ### Service Account Admission Controller The modification of pods is implemented via a plugin -called an [Admission Controller](admission-controllers.html). It is part of the apiserver. +called an [Admission Controller](admission-controllers). It is part of the apiserver. It acts synchronously to modify pods as they are created or updated. When this plugin is active (and it is by default on most distributions), then it does the following when a pod is created or modified: 1. If the pod does not have a `ServiceAccount` set, it sets the `ServiceAccount` to `default`. @@ -66,7 +62,7 @@ of type `ServiceAccountToken` with an annotation referencing the service account, and the controller will update it with a generated token: {% highlight json %} -{% raw %} + secret.json: { "kind": "Secret", @@ -79,22 +75,22 @@ secret.json: }, "type": "kubernetes.io/service-account-token" } -{% endraw %} + {% endhighlight %} {% highlight sh %} -{% raw %} + kubectl create -f ./secret.json kubectl describe secret mysecretname -{% endraw %} + {% endhighlight %} #### To delete/invalidate a service account token {% highlight sh %} -{% raw %} + kubectl delete secret mysecretname -{% endraw %} + {% endhighlight %} ### Service Account Controller diff --git a/v1.1/docs/admin/static-pods.md b/v1.1/docs/admin/static-pods.md index 8cfffd507d..d4898b2d5c 100644 --- a/v1.1/docs/admin/static-pods.md +++ b/v1.1/docs/admin/static-pods.md @@ -1,151 +1,130 @@ ---- -title: "Static pods (deprecated)" ---- - - -# Static pods (deprecated) - -**Static pods are to be deprecated and can be removed in any future Kubernetes release!** - -*Static pod* are managed directly by kubelet daemon on a specific node, without API server observing it. It does not have associated any replication controller, kubelet daemon itself watches it and restarts it when it crashes. There is no health check though. Static pods are always bound to one kubelet daemon and always run on the same node with it. - -Kubelet automatically creates so-called *mirror pod* on Kubernetes API server for each static pod, so the pods are visible there, but they cannot be controlled from the API server. - -## Static pod creation - -Static pod can be created in two ways: either by using configuration file(s) or by HTTP. - -### Configuration files - -The configuration files are just standard pod definition in json or yaml format in specific directory. Use `kubelet --config=` to start kubelet daemon, which periodically scans the directory and creates/deletes static pods as yaml/json files appear/disappear there. - -For example, this is how to start a simple web server as a static pod: - -1. Choose a node where we want to run the static pod. In this example, it's `my-minion1`. - -{% highlight console %} -{% raw %} - [joe@host ~] $ ssh my-minion1 -{% endraw %} -{% endhighlight %} - -2. Choose a directory, say `/etc/kubelet.d` and place a web server pod definition there, e.g. `/etc/kubernetes.d/static-web.yaml`: - -{% highlight console %} -{% raw %} - [root@my-minion1 ~] $ mkdir /etc/kubernetes.d/ - [root@my-minion1 ~] $ cat </etc/kubernetes.d/static-web.yaml - apiVersion: v1 - kind: Pod - metadata: - name: static-web - labels: - role: myrole - spec: - containers: - - name: web - image: nginx - ports: - - name: web - containerPort: 80 - protocol: tcp - EOF -{% endraw %} -{% endhighlight %} - -2. Configure your kubelet daemon on the node to use this directory by running it with `--config=/etc/kubelet.d/` argument. On Fedora Fedora 21 with Kubernetes 0.17 edit `/etc/kubernetes/kubelet` to include this line: - - ``` -{% raw %} - KUBELET_ARGS="--cluster-dns=10.254.0.10 --cluster-domain=kube.local --config=/etc/kubelet.d/" -{% endraw %} - ``` - - Instructions for other distributions or Kubernetes installations may vary. - -3. Restart kubelet. On Fedora 21, this is: - -{% highlight console %} -{% raw %} - [root@my-minion1 ~] $ systemctl restart kubelet -{% endraw %} -{% endhighlight %} - -## Pods created via HTTP - -Kubelet periodically downloads a file specified by `--manifest-url=` argument and interprets it as a json/yaml file with a pod definition. It works the same as `--config=`, i.e. it's reloaded every now and then and changes are applied to running static pods (see below). - -## Behavior of static pods - -When kubelet starts, it automatically starts all pods defined in directory specified in `--config=` or `--manifest-url=` arguments, i.e. our static-web. (It may take some time to pull nginx image, be patient…): - -{% highlight console %} -{% raw %} -[joe@my-minion1 ~] $ docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS NAMES -f6d05272b57e nginx:latest "nginx" 8 minutes ago Up 8 minutes k8s_web.6f802af4_static-web-fk-minion1_default_67e24ed9466ba55986d120c867395f3c_378e5f3c -{% endraw %} -{% endhighlight %} - -If we look at our Kubernetes API server (running on host `my-master`), we see that a new mirror-pod was created there too: - -{% highlight console %} -{% raw %} -[joe@host ~] $ ssh my-master -[joe@my-master ~] $ kubectl get pods -POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE -static-web-my-minion1 172.17.0.3 my-minion1/192.168.100.71 role=myrole Running 11 minutes - web nginx Running 11 minutes -{% endraw %} -{% endhighlight %} - -Labels from the static pod are propagated into the mirror-pod and can be used as usual for filtering. - -Notice we cannot delete the pod with the API server (e.g. via [`kubectl`](../user-guide/kubectl/kubectl.html) command), kubelet simply won't remove it. - -{% highlight console %} -{% raw %} -[joe@my-master ~] $ kubectl delete pod static-web-my-minion1 -pods/static-web-my-minion1 -[joe@my-master ~] $ kubectl get pods -POD IP CONTAINER(S) IMAGE(S) HOST ... -static-web-my-minion1 172.17.0.3 my-minion1/192.168.100.71 ... -{% endraw %} -{% endhighlight %} - -Back to our `my-minion1` host, we can try to stop the container manually and see, that kubelet automatically restarts it in a while: - -{% highlight console %} -{% raw %} -[joe@host ~] $ ssh my-minion1 -[joe@my-minion1 ~] $ docker stop f6d05272b57e -[joe@my-minion1 ~] $ sleep 20 -[joe@my-minion1 ~] $ docker ps -CONTAINER ID IMAGE COMMAND CREATED ... -5b920cbaf8b1 nginx:latest "nginx -g 'daemon of 2 seconds ago ... -{% endraw %} -{% endhighlight %} - -## Dynamic addition and removal of static pods - -Running kubelet periodically scans the configured directory (`/etc/kubelet.d` in our example) for changes and adds/removes pods as files appear/disappear in this directory. - -{% highlight console %} -{% raw %} -[joe@my-minion1 ~] $ mv /etc/kubernetes.d/static-web.yaml /tmp -[joe@my-minion1 ~] $ sleep 20 -[joe@my-minion1 ~] $ docker ps -// no nginx container is running -[joe@my-minion1 ~] $ mv /tmp/static-web.yaml /etc/kubernetes.d/ -[joe@my-minion1 ~] $ sleep 20 -[joe@my-minion1 ~] $ docker ps -CONTAINER ID IMAGE COMMAND CREATED ... -e7a62e3427f1 nginx:latest "nginx -g 'daemon of 27 seconds ago -{% endraw %} -{% endhighlight %} - - - - - - +--- +title: "Static pods (deprecated)" +--- + +**Static pods are to be deprecated and can be removed in any future Kubernetes release!** + +*Static pod* are managed directly by kubelet daemon on a specific node, without API server observing it. It does not have associated any replication controller, kubelet daemon itself watches it and restarts it when it crashes. There is no health check though. Static pods are always bound to one kubelet daemon and always run on the same node with it. + +Kubelet automatically creates so-called *mirror pod* on Kubernetes API server for each static pod, so the pods are visible there, but they cannot be controlled from the API server. + +## Static pod creation + +Static pod can be created in two ways: either by using configuration file(s) or by HTTP. + +### Configuration files + +The configuration files are just standard pod definition in json or yaml format in specific directory. Use `kubelet --config=` to start kubelet daemon, which periodically scans the directory and creates/deletes static pods as yaml/json files appear/disappear there. + +For example, this is how to start a simple web server as a static pod: + +1. Choose a node where we want to run the static pod. In this example, it's `my-minion1`. + +{% highlight console %} + [joe@host ~] $ ssh my-minion1 +{% endhighlight %} + +2. Choose a directory, say `/etc/kubelet.d` and place a web server pod definition there, e.g. `/etc/kubernetes.d/static-web.yaml`: + +{% highlight console %} + [root@my-minion1 ~] $ mkdir /etc/kubernetes.d/ + [root@my-minion1 ~] $ cat </etc/kubernetes.d/static-web.yaml + apiVersion: v1 + kind: Pod + metadata: + name: static-web + labels: + role: myrole + spec: + containers: + - name: web + image: nginx + ports: + - name: web + containerPort: 80 + protocol: tcp + EOF +{% endhighlight %} + +2. Configure your kubelet daemon on the node to use this directory by running it with `--config=/etc/kubelet.d/` argument. On Fedora Fedora 21 with Kubernetes 0.17 edit `/etc/kubernetes/kubelet` to include this line: + + ``` + KUBELET_ARGS="--cluster-dns=10.254.0.10 --cluster-domain=kube.local --config=/etc/kubelet.d/" + ``` + + Instructions for other distributions or Kubernetes installations may vary. + +3. Restart kubelet. On Fedora 21, this is: + +{% highlight console %} + [root@my-minion1 ~] $ systemctl restart kubelet +{% endhighlight %} + +## Pods created via HTTP + +Kubelet periodically downloads a file specified by `--manifest-url=` argument and interprets it as a json/yaml file with a pod definition. It works the same as `--config=`, i.e. it's reloaded every now and then and changes are applied to running static pods (see below). + +## Behavior of static pods + +When kubelet starts, it automatically starts all pods defined in directory specified in `--config=` or `--manifest-url=` arguments, i.e. our static-web. (It may take some time to pull nginx image, be patient'|): + +{% highlight console %} +[joe@my-minion1 ~] $ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS NAMES +f6d05272b57e nginx:latest "nginx" 8 minutes ago Up 8 minutes k8s_web.6f802af4_static-web-fk-minion1_default_67e24ed9466ba55986d120c867395f3c_378e5f3c +{% endhighlight %} + +If we look at our Kubernetes API server (running on host `my-master`), we see that a new mirror-pod was created there too: + +{% highlight console %} +[joe@host ~] $ ssh my-master +[joe@my-master ~] $ kubectl get pods +POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE +static-web-my-minion1 172.17.0.3 my-minion1/192.168.100.71 role=myrole Running 11 minutes + web nginx Running 11 minutes +{% endhighlight %} + +Labels from the static pod are propagated into the mirror-pod and can be used as usual for filtering. + +Notice we cannot delete the pod with the API server (e.g. via [`kubectl`](../user-guide/kubectl/kubectl) command), kubelet simply won't remove it. + +{% highlight console %} +[joe@my-master ~] $ kubectl delete pod static-web-my-minion1 +pods/static-web-my-minion1 +[joe@my-master ~] $ kubectl get pods +POD IP CONTAINER(S) IMAGE(S) HOST ... +static-web-my-minion1 172.17.0.3 my-minion1/192.168.100.71 ... +{% endhighlight %} + +Back to our `my-minion1` host, we can try to stop the container manually and see, that kubelet automatically restarts it in a while: + +{% highlight console %} +[joe@host ~] $ ssh my-minion1 +[joe@my-minion1 ~] $ docker stop f6d05272b57e +[joe@my-minion1 ~] $ sleep 20 +[joe@my-minion1 ~] $ docker ps +CONTAINER ID IMAGE COMMAND CREATED ... +5b920cbaf8b1 nginx:latest "nginx -g 'daemon of 2 seconds ago ... +{% endhighlight %} + +## Dynamic addition and removal of static pods + +Running kubelet periodically scans the configured directory (`/etc/kubelet.d` in our example) for changes and adds/removes pods as files appear/disappear in this directory. + +{% highlight console %} +[joe@my-minion1 ~] $ mv /etc/kubernetes.d/static-web.yaml /tmp +[joe@my-minion1 ~] $ sleep 20 +[joe@my-minion1 ~] $ docker ps +// no nginx container is running +[joe@my-minion1 ~] $ mv /tmp/static-web.yaml /etc/kubernetes.d/ +[joe@my-minion1 ~] $ sleep 20 +[joe@my-minion1 ~] $ docker ps +CONTAINER ID IMAGE COMMAND CREATED ... +e7a62e3427f1 nginx:latest "nginx -g 'daemon of 27 seconds ago +{% endhighlight %} + + + + + + diff --git a/v1.1/docs/api.md b/v1.1/docs/api.md index e0e58bcbd3..32d68964be 100644 --- a/v1.1/docs/api.md +++ b/v1.1/docs/api.md @@ -1,19 +1,15 @@ --- title: "The Kubernetes API" --- +Primary system and API concepts are documented in the [User guide](user-guide/README). - -# The Kubernetes API - -Primary system and API concepts are documented in the [User guide](user-guide/README.html). - -Overall API conventions are described in the [API conventions doc](devel/api-conventions.html). +Overall API conventions are described in the [API conventions doc](devel/api-conventions). Complete API details are documented via [Swagger](http://swagger.io/). The Kubernetes apiserver (aka "master") exports an API that can be used to retrieve the [Swagger spec](https://github.com/swagger-api/swagger-spec/tree/master/schemas/v1.2) for the Kubernetes API, by default at `/swaggerapi`, and a UI you can use to browse the API documentation at `/swagger-ui`. We also periodically update a [statically generated UI](http://kubernetes.io/third_party/swagger-ui/). -Remote access to the API is discussed in the [access doc](admin/accessing-the-api.html). +Remote access to the API is discussed in the [access doc](admin/accessing-the-api). -The Kubernetes API also serves as the foundation for the declarative configuration schema for the system. The [Kubectl](user-guide/kubectl/kubectl.html) command-line tool can be used to create, update, delete, and get API objects. +The Kubernetes API also serves as the foundation for the declarative configuration schema for the system. The [Kubectl](user-guide/kubectl/kubectl) command-line tool can be used to create, update, delete, and get API objects. Kubernetes also stores its serialized state (currently in [etcd](https://coreos.com/docs/distributed-configuration/getting-started-with-etcd/)) in terms of the API resources. @@ -23,7 +19,7 @@ Kubernetes itself is decomposed into multiple components, which interact through In our experience, any system that is successful needs to grow and change as new use cases emerge or existing ones change. Therefore, we expect the Kubernetes API to continuously change and grow. However, we intend to not break compatibility with existing clients, for an extended period of time. In general, new API resources and new resource fields can be expected to be added frequently. Elimination of resources or fields will require following a deprecation process. The precise deprecation policy for eliminating features is TBD, but once we reach our 1.0 milestone, there will be a specific policy. -What constitutes a compatible change and how to change the API are detailed by the [API change document](devel/api_changes.html). +What constitutes a compatible change and how to change the API are detailed by the [API change document](devel/api_changes). ## API versioning @@ -34,7 +30,7 @@ multiple API versions, each at a different API path, such as `/api/v1` or We chose to version at the API level rather than at the resource or field level to ensure that the API presents a clear, consistent view of system resources and behavior, and to enable controlling access to end-of-lifed and/or experimental APIs. Note that API versioning and Software versioning are only indirectly related. The [API and release -versioning proposal](design/versioning.html) describes the relationship between API versioning and +versioning proposal](design/versioning) describes the relationship between API versioning and software versioning. @@ -64,7 +60,7 @@ in more detail in the [API Changes documentation](devel/api_changes.html#alpha-b ## API groups To make it easier to extend the Kubernetes API, we are in the process of implementing [*API -groups*](proposals/api-group.html). These are simply different interfaces to read and/or modify the +groups*](proposals/api-group). These are simply different interfaces to read and/or modify the same underlying resources. The API group is specified in a REST path and in the `apiVersion` field of a serialized object. @@ -102,7 +98,7 @@ Changes to services are the most significant difference between v1beta3 and v1. Some other difference between v1beta3 and v1: -* The `pod.spec.containers[*].privileged` and `pod.spec.containers[*].capabilities` properties are now nested under the `pod.spec.containers[*].securityContext` property. See [Security Contexts](user-guide/security-context.html). +* The `pod.spec.containers[*].privileged` and `pod.spec.containers[*].capabilities` properties are now nested under the `pod.spec.containers[*].securityContext` property. See [Security Contexts](user-guide/security-context). * The `pod.spec.host` property is renamed to `pod.spec.nodeName`. * The `endpoints.subsets[*].addresses.IP` property is renamed to `endpoints.subsets[*].addresses.ip`. * The `pod.status.containerStatuses[*].state.termination` and `pod.status.containerStatuses[*].lastState.termination` properties are renamed to `pod.status.containerStatuses[*].state.terminated` and `pod.status.containerStatuses[*].lastState.terminated` respectively. diff --git a/v1.1/docs/design/README.md b/v1.1/docs/design/README.md index 8cfa3289ec..69a6362161 100644 --- a/v1.1/docs/design/README.md +++ b/v1.1/docs/design/README.md @@ -1,10 +1,6 @@ --- title: "Kubernetes Design Overview" --- - - -# Kubernetes Design Overview - Kubernetes is a system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications. Kubernetes establishes robust declarative primitives for maintaining the desired state requested by the user. We see these primitives as the main value added by Kubernetes. Self-healing mechanisms, such as auto-restarting, re-scheduling, and replicating containers require active controllers, not just imperative orchestration. @@ -15,11 +11,11 @@ Kubernetes enables users to ask a cluster to run a set of containers. The system Kubernetes is intended to run on a number of cloud providers, as well as on physical hosts. -A single Kubernetes cluster is not intended to span multiple availability zones. Instead, we recommend building a higher-level layer to replicate complete deployments of highly available applications across multiple zones (see [the multi-cluster doc](../admin/multi-cluster.html) and [cluster federation proposal](../proposals/federation.html) for more details). +A single Kubernetes cluster is not intended to span multiple availability zones. Instead, we recommend building a higher-level layer to replicate complete deployments of highly available applications across multiple zones (see [the multi-cluster doc](../admin/multi-cluster) and [cluster federation proposal](../proposals/federation) for more details). Finally, Kubernetes aspires to be an extensible, pluggable, building-block OSS platform and toolkit. Therefore, architecturally, we want Kubernetes to be built as a collection of pluggable components and layers, with the ability to use alternative schedulers, controllers, storage systems, and distribution mechanisms, and we're evolving its current code in that direction. Furthermore, we want others to be able to extend Kubernetes functionality, such as with higher-level PaaS functionality or multi-cluster layers, without modification of core Kubernetes source. Therefore, its API isn't just (or even necessarily mainly) targeted at end users, but at tool and extension developers. Its APIs are intended to serve as the foundation for an open ecosystem of tools, automation systems, and higher-level API layers. Consequently, there are no "internal" inter-component APIs. All APIs are visible and available, including the APIs used by the scheduler, the node controller, the replication-controller manager, Kubelet's API, etc. There's no glass to break -- in order to handle more complex use cases, one can just access the lower-level APIs in a fully transparent, composable manner. -For more about the Kubernetes architecture, see [architecture](architecture.html). +For more about the Kubernetes architecture, see [architecture](architecture). diff --git a/v1.1/docs/design/access.md b/v1.1/docs/design/access.md index 9bea9ed05a..74b0366de8 100644 --- a/v1.1/docs/design/access.md +++ b/v1.1/docs/design/access.md @@ -1,13 +1,8 @@ --- title: "K8s Identity and Access Management Sketch" --- - - -# K8s Identity and Access Management Sketch - This document suggests a direction for identity and access management in the Kubernetes system. - ## Background High level goals are: @@ -154,7 +149,7 @@ Improvements: K8s will have a have a `namespace` API object. It is similar to a Google Compute Engine `project`. It provides a namespace for objects created by a group of people co-operating together, preventing name collisions with non-cooperating groups. It also serves as a reference point for authorization policies. -Namespaces are described in [namespaces.md](namespaces.html). +Namespaces are described in [namespaces.md](namespaces). In the Enterprise Profile: - a `userAccount` may have permission to access several `namespace`s. @@ -164,7 +159,7 @@ In the Simple Profile: Namespaces versus userAccount vs Labels: - `userAccount`s are intended for audit logging (both name and UID should be logged), and to define who has access to `namespace`s. -- `labels` (see [docs/user-guide/labels.md](../../docs/user-guide/labels.html)) should be used to distinguish pods, users, and other objects that cooperate towards a common goal but are different in some way, such as version, or responsibilities. +- `labels` (see [docs/user-guide/labels.md](/{{page.version}}/docs/user-guide/labels)) should be used to distinguish pods, users, and other objects that cooperate towards a common goal but are different in some way, such as version, or responsibilities. - `namespace`s prevent name collisions between uncoordinated groups of people, and provide a place to attach common policies for co-operating groups of people. @@ -225,7 +220,7 @@ Policy objects may be applicable only to a single namespace or to all namespaces ## Accounting -The API should have a `quota` concept (see http://issue.k8s.io/442). A quota object relates a namespace (and optionally a label selector) to a maximum quantity of resources that may be used (see [resources design doc](resources.html)). +The API should have a `quota` concept (see http://issue.k8s.io/442). A quota object relates a namespace (and optionally a label selector) to a maximum quantity of resources that may be used (see [resources design doc](resources)). Initially: - a `quota` object is immutable. diff --git a/v1.1/docs/design/admission_control.md b/v1.1/docs/design/admission_control.md index fbf19d1a62..959c4fffc1 100644 --- a/v1.1/docs/design/admission_control.md +++ b/v1.1/docs/design/admission_control.md @@ -1,10 +1,6 @@ --- title: "Kubernetes Proposal - Admission Control" --- - - -# Kubernetes Proposal - Admission Control - **Related PR:** | Topic | Link | @@ -41,7 +37,7 @@ The kube-apiserver takes the following OPTIONAL arguments to enable admission co An **AdmissionControl** plug-in is an implementation of the following interface: {% highlight go %} -{% raw %} + package admission // Attributes is an interface used by a plug-in to make an admission decision on a individual request. @@ -58,18 +54,18 @@ type Interface interface { // An error is returned if it denies the request. Admit(a Attributes) (err error) } -{% endraw %} + {% endhighlight %} A **plug-in** must be compiled with the binary, and is registered as an available option by providing a name, and implementation of admission.Interface. {% highlight go %} -{% raw %} + func init() { admission.RegisterPlugin("AlwaysDeny", func(client client.Interface, config io.Reader) (admission.Interface, error) { return NewAlwaysDeny(), nil }) } -{% endraw %} + {% endhighlight %} Invocation of admission control is handled by the **APIServer** and not individual **RESTStorage** implementations. diff --git a/v1.1/docs/design/admission_control_limit_range.md b/v1.1/docs/design/admission_control_limit_range.md index 677d25af83..1c51cca67c 100644 --- a/v1.1/docs/design/admission_control_limit_range.md +++ b/v1.1/docs/design/admission_control_limit_range.md @@ -1,10 +1,6 @@ --- title: "Admission control plugin: LimitRanger" --- - - -# Admission control plugin: LimitRanger - ## Background This document proposes a system for enforcing resource requirements constraints as part of admission control. @@ -25,7 +21,7 @@ The **LimitRange** resource is scoped to a **Namespace**. ### Type {% highlight go %} -{% raw %} + // LimitType is a type of object that is limited type LimitType string @@ -81,7 +77,7 @@ type LimitRangeList struct { // More info: http://releases.k8s.io/release-1.1/docs/design/admission_control_limit_range.md Items []LimitRange `json:"items"` } -{% endraw %} + {% endhighlight %} ### Validation @@ -95,21 +91,21 @@ Min (if specified) <= DefaultRequest (if specified) <= Default (if specified) <= The following default value behaviors are applied to a LimitRange for a given named resource. ``` -{% raw %} + if LimitRangeItem.Default[resourceName] is undefined if LimitRangeItem.Max[resourceName] is defined LimitRangeItem.Default[resourceName] = LimitRangeItem.Max[resourceName] -{% endraw %} + ``` ``` -{% raw %} + if LimitRangeItem.DefaultRequest[resourceName] is undefined if LimitRangeItem.Default[resourceName] is defined LimitRangeItem.DefaultRequest[resourceName] = LimitRangeItem.Default[resourceName] else if LimitRangeItem.Min[resourceName] is defined LimitRangeItem.DefaultRequest[resourceName] = LimitRangeItem.Min[resourceName] -{% endraw %} + ``` ## AdmissionControl plugin: LimitRanger @@ -121,9 +117,9 @@ If a constraint is not specified for an enumerated resource, it is not enforced To enable the plug-in and support for LimitRange, the kube-apiserver must be configured as follows: {% highlight console %} -{% raw %} + $ kube-apiserver --admission-control=LimitRanger -{% endraw %} + {% endhighlight %} ### Enforcement of constraints @@ -172,7 +168,7 @@ Across all containers in pod, the following must hold true The default ```LimitRange``` that is applied via Salt configuration will be updated as follows: ``` -{% raw %} + apiVersion: "v1" kind: "LimitRange" metadata: @@ -183,7 +179,7 @@ spec: - type: "Container" defaultRequests: cpu: "100m" -{% endraw %} + ``` ## Example diff --git a/v1.1/docs/design/admission_control_resource_quota.md b/v1.1/docs/design/admission_control_resource_quota.md index b196d81095..1381e30e76 100644 --- a/v1.1/docs/design/admission_control_resource_quota.md +++ b/v1.1/docs/design/admission_control_resource_quota.md @@ -1,10 +1,6 @@ --- title: "Admission control plugin: ResourceQuota" --- - - -# Admission control plugin: ResourceQuota - ## Background This document describes a system for enforcing hard resource usage limits per namespace as part of admission control. @@ -20,7 +16,7 @@ This document describes a system for enforcing hard resource usage limits per na The **ResourceQuota** object is scoped to a **Namespace**. {% highlight go %} -{% raw %} + // The following identify resource constants for Kubernetes object types const ( // Pods, number @@ -71,7 +67,7 @@ type ResourceQuotaList struct { // Items is a list of ResourceQuota objects Items []ResourceQuota `json:"items" description:"items is a list of ResourceQuota objects; see http://releases.k8s.io/release-1.1/docs/design/admission_control_resource_quota.md#admissioncontrol-plugin-resourcequota"` } -{% endraw %} + {% endhighlight %} ## Quota Tracked Resources @@ -152,9 +148,9 @@ The **ResourceQuota** plug-in introspects all incoming admission requests. To enable the plug-in and support for ResourceQuota, the kube-apiserver must be configured as follows: ``` -{% raw %} + $ kube-apiserver --admission-control=ResourceQuota -{% endraw %} + ``` It makes decisions by evaluating the incoming object against all defined **ResourceQuota.Status.Hard** resource limits in the request @@ -177,7 +173,7 @@ kubectl is modified to support the **ResourceQuota** resource. For example, {% highlight console %} -{% raw %} + $ kubectl create -f docs/admin/resourcequota/namespace.yaml namespace "quota-example" created $ kubectl create -f docs/admin/resourcequota/quota.yaml --namespace=quota-example @@ -195,11 +191,11 @@ replicationcontrollers 0 20 resourcequotas 1 1 secrets 1 10 services 0 5 -{% endraw %} + {% endhighlight %} ## More information -See [resource quota document](../admin/resource-quota.html) and the [example of Resource Quota](../admin/resourcequota/) for more information. +See [resource quota document](../admin/resource-quota) and the [example of Resource Quota](../admin/resourcequota/) for more information. diff --git a/v1.1/docs/design/architecture.md b/v1.1/docs/design/architecture.md index 2a5affdef8..d814bc9379 100644 --- a/v1.1/docs/design/architecture.md +++ b/v1.1/docs/design/architecture.md @@ -1,10 +1,6 @@ --- title: "Kubernetes architecture" --- - - -# Kubernetes architecture - A running Kubernetes cluster contains node agents (`kubelet`) and master components (APIs, scheduler, etc), on top of a distributed storage solution. This diagram shows our desired eventual state, though we're still working on a few things, like making `kubelet` itself (all our components, really) run within containers, and making the scheduler 100% pluggable. ![Architecture Diagram](architecture.png?raw=true "Architecture overview") @@ -19,13 +15,13 @@ Each node runs Docker, of course. Docker takes care of the details of downloadi ### `kubelet` -The `kubelet` manages [pods](../user-guide/pods.html) and their containers, their images, their volumes, etc. +The `kubelet` manages [pods](../user-guide/pods) and their containers, their images, their volumes, etc. ### `kube-proxy` -Each node also runs a simple network proxy and load balancer (see the [services FAQ](https://github.com/kubernetes/kubernetes/wiki/Services-FAQ) for more details). This reflects `services` (see [the services doc](../user-guide/services.html) for more details) as defined in the Kubernetes API on each node and can do simple TCP and UDP stream forwarding (round robin) across a set of backends. +Each node also runs a simple network proxy and load balancer (see the [services FAQ](https://github.com/kubernetes/kubernetes/wiki/Services-FAQ) for more details). This reflects `services` (see [the services doc](../user-guide/services) for more details) as defined in the Kubernetes API on each node and can do simple TCP and UDP stream forwarding (round robin) across a set of backends. -Service endpoints are currently found via [DNS](../admin/dns.html) or through environment variables (both [Docker-links-compatible](https://docs.docker.com/userguide/dockerlinks/) and Kubernetes `{FOO}_SERVICE_HOST` and `{FOO}_SERVICE_PORT` variables are supported). These variables resolve to ports managed by the service proxy. +Service endpoints are currently found via [DNS](../admin/dns) or through environment variables (both [Docker-links-compatible](https://docs.docker.com/userguide/dockerlinks/) and Kubernetes `{FOO}_SERVICE_HOST` and `{FOO}_SERVICE_PORT` variables are supported). These variables resolve to ports managed by the service proxy. ## The Kubernetes Control Plane @@ -37,7 +33,7 @@ All persistent master state is stored in an instance of `etcd`. This provides a ### Kubernetes API Server -The apiserver serves up the [Kubernetes API](../api.html). It is intended to be a CRUD-y server, with most/all business logic implemented in separate components or in plug-ins. It mainly processes REST operations, validates them, and updates the corresponding objects in `etcd` (and eventually other stores). +The apiserver serves up the [Kubernetes API](../api). It is intended to be a CRUD-y server, with most/all business logic implemented in separate components or in plug-ins. It mainly processes REST operations, validates them, and updates the corresponding objects in `etcd` (and eventually other stores). ### Scheduler @@ -47,7 +43,7 @@ The scheduler binds unscheduled pods to nodes via the `/binding` API. The schedu All other cluster-level functions are currently performed by the Controller Manager. For instance, `Endpoints` objects are created and updated by the endpoints controller, and nodes are discovered, managed, and monitored by the node controller. These could eventually be split into separate components to make them independently pluggable. -The [`replicationcontroller`](../user-guide/replication-controller.html) is a mechanism that is layered on top of the simple [`pod`](../user-guide/pods.html) API. We eventually plan to port it to a generic plug-in mechanism, once one is implemented. +The [`replicationcontroller`](../user-guide/replication-controller) is a mechanism that is layered on top of the simple [`pod`](../user-guide/pods) API. We eventually plan to port it to a generic plug-in mechanism, once one is implemented. diff --git a/v1.1/docs/design/clustering.md b/v1.1/docs/design/clustering.md index d91b3c8890..38056fd482 100644 --- a/v1.1/docs/design/clustering.md +++ b/v1.1/docs/design/clustering.md @@ -1,11 +1,6 @@ --- title: "Clustering in Kubernetes" --- - - -# Clustering in Kubernetes - - ## Overview The term "clustering" refers to the process of having all members of the Kubernetes cluster find and trust each other. There are multiple different ways to achieve clustering with different security and usability profiles. This document attempts to lay out the user experiences for clustering that Kubernetes aims to address. diff --git a/v1.1/docs/design/clustering/README.md b/v1.1/docs/design/clustering/README.md index b37d22c02c..b182dd4f00 100644 --- a/v1.1/docs/design/clustering/README.md +++ b/v1.1/docs/design/clustering/README.md @@ -1,38 +1,34 @@ ---- -title: "Building with Docker" ---- - -This directory contains diagrams for the clustering design doc. - -This depends on the `seqdiag` [utility](http://blockdiag.com/en/seqdiag/index.html). Assuming you have a non-borked python install, this should be installable with - -{% highlight sh %} -{% raw %} -pip install seqdiag -{% endraw %} -{% endhighlight %} - -Just call `make` to regenerate the diagrams. - -## Building with Docker - -If you are on a Mac or your pip install is messed up, you can easily build with docker. - -{% highlight sh %} -{% raw %} -make docker -{% endraw %} -{% endhighlight %} - -The first run will be slow but things should be fast after that. - -To clean up the docker containers that are created (and other cruft that is left around) you can run `make docker-clean`. - -If you are using boot2docker and get warnings about clock skew (or if things aren't building for some reason) then you can fix that up with `make fix-clock-skew`. - -## Automatically rebuild on file changes - -If you have the fswatch utility installed, you can have it monitor the file system and automatically rebuild when files have changed. Just do a `make watch`. - - - +--- +title: "Building with Docker" +--- + +This directory contains diagrams for the clustering design doc. + +This depends on the `seqdiag` [utility](http://blockdiag.com/en/seqdiag/index). Assuming you have a non-borked python install, this should be installable with + +{% highlight sh %} +pip install seqdiag +{% endhighlight %} + +Just call `make` to regenerate the diagrams. + +## Building with Docker + +If you are on a Mac or your pip install is messed up, you can easily build with docker. + +{% highlight sh %} +make docker +{% endhighlight %} + +The first run will be slow but things should be fast after that. + +To clean up the docker containers that are created (and other cruft that is left around) you can run `make docker-clean`. + +If you are using boot2docker and get warnings about clock skew (or if things aren't building for some reason) then you can fix that up with `make fix-clock-skew`. + +## Automatically rebuild on file changes + +If you have the fswatch utility installed, you can have it monitor the file system and automatically rebuild when files have changed. Just do a `make watch`. + + + diff --git a/v1.1/docs/design/clustering/index.md b/v1.1/docs/design/clustering/index.md index b37d22c02c..b182dd4f00 100644 --- a/v1.1/docs/design/clustering/index.md +++ b/v1.1/docs/design/clustering/index.md @@ -1,38 +1,34 @@ ---- -title: "Building with Docker" ---- - -This directory contains diagrams for the clustering design doc. - -This depends on the `seqdiag` [utility](http://blockdiag.com/en/seqdiag/index.html). Assuming you have a non-borked python install, this should be installable with - -{% highlight sh %} -{% raw %} -pip install seqdiag -{% endraw %} -{% endhighlight %} - -Just call `make` to regenerate the diagrams. - -## Building with Docker - -If you are on a Mac or your pip install is messed up, you can easily build with docker. - -{% highlight sh %} -{% raw %} -make docker -{% endraw %} -{% endhighlight %} - -The first run will be slow but things should be fast after that. - -To clean up the docker containers that are created (and other cruft that is left around) you can run `make docker-clean`. - -If you are using boot2docker and get warnings about clock skew (or if things aren't building for some reason) then you can fix that up with `make fix-clock-skew`. - -## Automatically rebuild on file changes - -If you have the fswatch utility installed, you can have it monitor the file system and automatically rebuild when files have changed. Just do a `make watch`. - - - +--- +title: "Building with Docker" +--- + +This directory contains diagrams for the clustering design doc. + +This depends on the `seqdiag` [utility](http://blockdiag.com/en/seqdiag/index). Assuming you have a non-borked python install, this should be installable with + +{% highlight sh %} +pip install seqdiag +{% endhighlight %} + +Just call `make` to regenerate the diagrams. + +## Building with Docker + +If you are on a Mac or your pip install is messed up, you can easily build with docker. + +{% highlight sh %} +make docker +{% endhighlight %} + +The first run will be slow but things should be fast after that. + +To clean up the docker containers that are created (and other cruft that is left around) you can run `make docker-clean`. + +If you are using boot2docker and get warnings about clock skew (or if things aren't building for some reason) then you can fix that up with `make fix-clock-skew`. + +## Automatically rebuild on file changes + +If you have the fswatch utility installed, you can have it monitor the file system and automatically rebuild when files have changed. Just do a `make watch`. + + + diff --git a/v1.1/docs/design/command_execution_port_forwarding.md b/v1.1/docs/design/command_execution_port_forwarding.md index 2fe5dfd556..8552985c4a 100644 --- a/v1.1/docs/design/command_execution_port_forwarding.md +++ b/v1.1/docs/design/command_execution_port_forwarding.md @@ -1,10 +1,6 @@ --- title: "Container Command Execution & Port Forwarding in Kubernetes" --- - - -# Container Command Execution & Port Forwarding in Kubernetes - ## Abstract This describes an approach for providing support for: diff --git a/v1.1/docs/design/daemon.md b/v1.1/docs/design/daemon.md index 8a1824b642..7ce790c271 100644 --- a/v1.1/docs/design/daemon.md +++ b/v1.1/docs/design/daemon.md @@ -1,131 +1,125 @@ ---- -title: "DaemonSet in Kubernetes" ---- - - -# DaemonSet in Kubernetes - -**Author**: Ananya Kumar (@AnanyaKumar) - -**Status**: Implemented. - -This document presents the design of the Kubernetes DaemonSet, describes use cases, and gives an overview of the code. - -## Motivation - -Many users have requested for a way to run a daemon on every node in a Kubernetes cluster, or on a certain set of nodes in a cluster. This is essential for use cases such as building a sharded datastore, or running a logger on every node. In comes the DaemonSet, a way to conveniently create and manage daemon-like workloads in Kubernetes. - -## Use Cases - -The DaemonSet can be used for user-specified system services, cluster-level applications with strong node ties, and Kubernetes node services. Below are example use cases in each category. - -### User-Specified System Services: - -Logging: Some users want a way to collect statistics about nodes in a cluster and send those logs to an external database. For example, system administrators might want to know if their machines are performing as expected, if they need to add more machines to the cluster, or if they should switch cloud providers. The DaemonSet can be used to run a data collection service (for example fluentd) on every node and send the data to a service like ElasticSearch for analysis. - -### Cluster-Level Applications - -Datastore: Users might want to implement a sharded datastore in their cluster. A few nodes in the cluster, labeled ‘app=datastore’, might be responsible for storing data shards, and pods running on these nodes might serve data. This architecture requires a way to bind pods to specific nodes, so it cannot be achieved using a Replication Controller. A DaemonSet is a convenient way to implement such a datastore. - -For other uses, see the related [feature request](https://issues.k8s.io/1518) - -## Functionality - -The DaemonSet supports standard API features: -- create - - The spec for DaemonSets has a pod template field. - - Using the pod’s nodeSelector field, DaemonSets can be restricted to operate over nodes that have a certain label. For example, suppose that in a cluster some nodes are labeled ‘app=database’. You can use a DaemonSet to launch a datastore pod on exactly those nodes labeled ‘app=database’. - - Using the pod's nodeName field, DaemonSets can be restricted to operate on a specified node. - - The PodTemplateSpec used by the DaemonSet is the same as the PodTemplateSpec used by the Replication Controller. - - The initial implementation will not guarnatee that DaemonSet pods are created on nodes before other pods. - - The initial implementation of DaemonSet does not guarantee that DaemonSet pods show up on nodes (for example because of resource limitations of the node), but makes a best effort to launch DaemonSet pods (like Replication Controllers do with pods). Subsequent revisions might ensure that DaemonSet pods show up on nodes, preempting other pods if necessary. - - The DaemonSet controller adds an annotation "kubernetes.io/created-by: \" - - YAML example: - -{% highlight yaml %} -{% raw %} - apiVersion: v1 - kind: DaemonSet - metadata: - labels: - app: datastore - name: datastore - spec: - template: - metadata: - labels: - app: datastore-shard - spec: - nodeSelector: - app: datastore-node - containers: - name: datastore-shard - image: kubernetes/sharded - ports: - - containerPort: 9042 - name: main -{% endraw %} -{% endhighlight %} - - - commands that get info - - get (e.g. kubectl get daemonsets) - - describe - - Modifiers - - delete (if --cascade=true, then first the client turns down all the pods controlled by the DaemonSet (by setting the nodeSelector to a uuid pair that is unlikely to be set on any node); then it deletes the DaemonSet; then it deletes the pods) - - label - - annotate - - update operations like patch and replace (only allowed to selector and to nodeSelector and nodeName of pod template) - - DaemonSets have labels, so you could, for example, list all DaemonSets with certain labels (the same way you would for a Replication Controller). - - In general, for all the supported features like get, describe, update, etc, the DaemonSet works in a similar way to the Replication Controller. However, note that the DaemonSet and the Replication Controller are different constructs. - -### Persisting Pods - - - Ordinary liveness probes specified in the pod template work to keep pods created by a DaemonSet running. - - If a daemon pod is killed or stopped, the DaemonSet will create a new replica of the daemon pod on the node. - -### Cluster Mutations - - - When a new node is added to the cluster, the DaemonSet controller starts daemon pods on the node for DaemonSets whose pod template nodeSelectors match the node’s labels. - - Suppose the user launches a DaemonSet that runs a logging daemon on all nodes labeled “logger=fluentdâ€?. If the user then adds the “logger=fluentdâ€? label to a node (that did not initially have the label), the logging daemon will launch on the node. Additionally, if a user removes the label from a node, the logging daemon on that node will be killed. - -## Alternatives Considered - -We considered several alternatives, that were deemed inferior to the approach of creating a new DaemonSet abstraction. - -One alternative is to include the daemon in the machine image. In this case it would run outside of Kubernetes proper, and thus not be monitored, health checked, usable as a service endpoint, easily upgradable, etc. - -A related alternative is to package daemons as static pods. This would address most of the problems described above, but they would still not be easily upgradable, and more generally could not be managed through the API server interface. - -A third alternative is to generalize the Replication Controller. We would do something like: if you set the `replicas` field of the ReplicationConrollerSpec to -1, then it means "run exactly one replica on every node matching the nodeSelector in the pod template." The ReplicationController would pretend `replicas` had been set to some large number -- larger than the largest number of nodes ever expected in the cluster -- and would use some anti-affinity mechanism to ensure that no more than one Pod from the ReplicationController runs on any given node. There are two downsides to this approach. First, there would always be a large number of Pending pods in the scheduler (these will be scheduled onto new machines when they are added to the cluster). The second downside is more philosophical: DaemonSet and the Replication Controller are very different concepts. We believe that having small, targeted controllers for distinct purposes makes Kubernetes easier to understand and use, compared to having larger multi-functional controllers (see ["Convert ReplicationController to a plugin"](http://issues.k8s.io/3058) for some discussion of this topic). - -## Design - -#### Client - -- Add support for DaemonSet commands to kubectl and the client. Client code was added to client/unversioned. The main files in Kubectl that were modified are kubectl/describe.go and kubectl/stop.go, since for other calls like Get, Create, and Update, the client simply forwards the request to the backend via the REST API. - -#### Apiserver - -- Accept, parse, validate client commands -- REST API calls are handled in registry/daemon - - In particular, the api server will add the object to etcd - - DaemonManager listens for updates to etcd (using Framework.informer) -- API objects for DaemonSet were created in expapi/v1/types.go and expapi/v1/register.go -- Validation code is in expapi/validation - -#### Daemon Manager - -- Creates new DaemonSets when requested. Launches the corresponding daemon pod on all nodes with labels matching the new DaemonSet’s selector. -- Listens for addition of new nodes to the cluster, by setting up a framework.NewInformer that watches for the creation of Node API objects. When a new node is added, the daemon manager will loop through each DaemonSet. If the label of the node matches the selector of the DaemonSet, then the daemon manager will create the corresponding daemon pod in the new node. -- The daemon manager creates a pod on a node by sending a command to the API server, requesting for a pod to be bound to the node (the node will be specified via its hostname) - -#### Kubelet - -- Does not need to be modified, but health checking will occur for the daemon pods and revive the pods if they are killed (we set the pod restartPolicy to Always). We reject DaemonSet objects with pod templates that don’t have restartPolicy set to Always. - -## Open Issues - -- Should work similarly to [Deployment](http://issues.k8s.io/1743). - - - +--- +title: "DaemonSet in Kubernetes" +--- +**Author**: Ananya Kumar (@AnanyaKumar) + +**Status**: Implemented. + +This document presents the design of the Kubernetes DaemonSet, describes use cases, and gives an overview of the code. + +## Motivation + +Many users have requested for a way to run a daemon on every node in a Kubernetes cluster, or on a certain set of nodes in a cluster. This is essential for use cases such as building a sharded datastore, or running a logger on every node. In comes the DaemonSet, a way to conveniently create and manage daemon-like workloads in Kubernetes. + +## Use Cases + +The DaemonSet can be used for user-specified system services, cluster-level applications with strong node ties, and Kubernetes node services. Below are example use cases in each category. + +### User-Specified System Services: + +Logging: Some users want a way to collect statistics about nodes in a cluster and send those logs to an external database. For example, system administrators might want to know if their machines are performing as expected, if they need to add more machines to the cluster, or if they should switch cloud providers. The DaemonSet can be used to run a data collection service (for example fluentd) on every node and send the data to a service like ElasticSearch for analysis. + +### Cluster-Level Applications + +Datastore: Users might want to implement a sharded datastore in their cluster. A few nodes in the cluster, labeled 'app=datastore', might be responsible for storing data shards, and pods running on these nodes might serve data. This architecture requires a way to bind pods to specific nodes, so it cannot be achieved using a Replication Controller. A DaemonSet is a convenient way to implement such a datastore. + +For other uses, see the related [feature request](https://issues.k8s.io/1518) + +## Functionality + +The DaemonSet supports standard API features: +- create + - The spec for DaemonSets has a pod template field. + - Using the pod's nodeSelector field, DaemonSets can be restricted to operate over nodes that have a certain label. For example, suppose that in a cluster some nodes are labeled 'app=database'. You can use a DaemonSet to launch a datastore pod on exactly those nodes labeled 'app=database'. + - Using the pod's nodeName field, DaemonSets can be restricted to operate on a specified node. + - The PodTemplateSpec used by the DaemonSet is the same as the PodTemplateSpec used by the Replication Controller. + - The initial implementation will not guarnatee that DaemonSet pods are created on nodes before other pods. + - The initial implementation of DaemonSet does not guarantee that DaemonSet pods show up on nodes (for example because of resource limitations of the node), but makes a best effort to launch DaemonSet pods (like Replication Controllers do with pods). Subsequent revisions might ensure that DaemonSet pods show up on nodes, preempting other pods if necessary. + - The DaemonSet controller adds an annotation "kubernetes.io/created-by: \" + - YAML example: + +{% highlight yaml %} + apiVersion: v1 + kind: DaemonSet + metadata: + labels: + app: datastore + name: datastore + spec: + template: + metadata: + labels: + app: datastore-shard + spec: + nodeSelector: + app: datastore-node + containers: + name: datastore-shard + image: kubernetes/sharded + ports: + - containerPort: 9042 + name: main +{% endhighlight %} + + - commands that get info + - get (e.g. kubectl get daemonsets) + - describe + - Modifiers + - delete (if --cascade=true, then first the client turns down all the pods controlled by the DaemonSet (by setting the nodeSelector to a uuid pair that is unlikely to be set on any node); then it deletes the DaemonSet; then it deletes the pods) + - label + - annotate + - update operations like patch and replace (only allowed to selector and to nodeSelector and nodeName of pod template) + - DaemonSets have labels, so you could, for example, list all DaemonSets with certain labels (the same way you would for a Replication Controller). + - In general, for all the supported features like get, describe, update, etc, the DaemonSet works in a similar way to the Replication Controller. However, note that the DaemonSet and the Replication Controller are different constructs. + +### Persisting Pods + + - Ordinary liveness probes specified in the pod template work to keep pods created by a DaemonSet running. + - If a daemon pod is killed or stopped, the DaemonSet will create a new replica of the daemon pod on the node. + +### Cluster Mutations + + - When a new node is added to the cluster, the DaemonSet controller starts daemon pods on the node for DaemonSets whose pod template nodeSelectors match the node's labels. + - Suppose the user launches a DaemonSet that runs a logging daemon on all nodes labeled 'logger=fluentd'?. If the user then adds the 'logger=fluentd'? label to a node (that did not initially have the label), the logging daemon will launch on the node. Additionally, if a user removes the label from a node, the logging daemon on that node will be killed. + +## Alternatives Considered + +We considered several alternatives, that were deemed inferior to the approach of creating a new DaemonSet abstraction. + +One alternative is to include the daemon in the machine image. In this case it would run outside of Kubernetes proper, and thus not be monitored, health checked, usable as a service endpoint, easily upgradable, etc. + +A related alternative is to package daemons as static pods. This would address most of the problems described above, but they would still not be easily upgradable, and more generally could not be managed through the API server interface. + +A third alternative is to generalize the Replication Controller. We would do something like: if you set the `replicas` field of the ReplicationConrollerSpec to -1, then it means "run exactly one replica on every node matching the nodeSelector in the pod template." The ReplicationController would pretend `replicas` had been set to some large number -- larger than the largest number of nodes ever expected in the cluster -- and would use some anti-affinity mechanism to ensure that no more than one Pod from the ReplicationController runs on any given node. There are two downsides to this approach. First, there would always be a large number of Pending pods in the scheduler (these will be scheduled onto new machines when they are added to the cluster). The second downside is more philosophical: DaemonSet and the Replication Controller are very different concepts. We believe that having small, targeted controllers for distinct purposes makes Kubernetes easier to understand and use, compared to having larger multi-functional controllers (see ["Convert ReplicationController to a plugin"](http://issues.k8s.io/3058) for some discussion of this topic). + +## Design + +#### Client + +- Add support for DaemonSet commands to kubectl and the client. Client code was added to client/unversioned. The main files in Kubectl that were modified are kubectl/describe.go and kubectl/stop.go, since for other calls like Get, Create, and Update, the client simply forwards the request to the backend via the REST API. + +#### Apiserver + +- Accept, parse, validate client commands +- REST API calls are handled in registry/daemon + - In particular, the api server will add the object to etcd + - DaemonManager listens for updates to etcd (using Framework.informer) +- API objects for DaemonSet were created in expapi/v1/types.go and expapi/v1/register.go +- Validation code is in expapi/validation + +#### Daemon Manager + +- Creates new DaemonSets when requested. Launches the corresponding daemon pod on all nodes with labels matching the new DaemonSet's selector. +- Listens for addition of new nodes to the cluster, by setting up a framework.NewInformer that watches for the creation of Node API objects. When a new node is added, the daemon manager will loop through each DaemonSet. If the label of the node matches the selector of the DaemonSet, then the daemon manager will create the corresponding daemon pod in the new node. +- The daemon manager creates a pod on a node by sending a command to the API server, requesting for a pod to be bound to the node (the node will be specified via its hostname) + +#### Kubelet + +- Does not need to be modified, but health checking will occur for the daemon pods and revive the pods if they are killed (we set the pod restartPolicy to Always). We reject DaemonSet objects with pod templates that don't have restartPolicy set to Always. + +## Open Issues + +- Should work similarly to [Deployment](http://issues.k8s.io/1743). + + + diff --git a/v1.1/docs/design/event_compression.md b/v1.1/docs/design/event_compression.md index c1278b85c1..2002936c45 100644 --- a/v1.1/docs/design/event_compression.md +++ b/v1.1/docs/design/event_compression.md @@ -1,10 +1,6 @@ --- title: "Kubernetes Event Compression" --- - - -# Kubernetes Event Compression - This document captures the design of event compression. @@ -63,7 +59,7 @@ Each binary that generates events: Sample kubectl output {% highlight console %} -{% raw %} + FIRSTSEEN LASTSEEN COUNT NAME KIND SUBOBJECT REASON SOURCE MESSAGE Thu, 12 Feb 2015 01:13:02 +0000 Thu, 12 Feb 2015 01:13:02 +0000 1 kubernetes-minion-4.c.saad-dev-vms.internal Minion starting {kubelet kubernetes-minion-4.c.saad-dev-vms.internal} Starting kubelet. Thu, 12 Feb 2015 01:13:09 +0000 Thu, 12 Feb 2015 01:13:09 +0000 1 kubernetes-minion-1.c.saad-dev-vms.internal Minion starting {kubelet kubernetes-minion-1.c.saad-dev-vms.internal} Starting kubelet. @@ -76,7 +72,7 @@ Thu, 12 Feb 2015 01:13:05 +0000 Thu, 12 Feb 2015 01:13:12 +0000 4 Thu, 12 Feb 2015 01:13:05 +0000 Thu, 12 Feb 2015 01:13:12 +0000 4 monitoring-heapster-controller-oh43e Pod failedScheduling {scheduler } Error scheduling: no nodes available to schedule pods Thu, 12 Feb 2015 01:13:20 +0000 Thu, 12 Feb 2015 01:13:20 +0000 1 kibana-logging-controller-gziey BoundPod implicitly required container POD pulled {kubelet kubernetes-minion-4.c.saad-dev-vms.internal} Successfully pulled image "kubernetes/pause:latest" Thu, 12 Feb 2015 01:13:20 +0000 Thu, 12 Feb 2015 01:13:20 +0000 1 kibana-logging-controller-gziey Pod scheduled {scheduler } Successfully assigned kibana-logging-controller-gziey to kubernetes-minion-4.c.saad-dev-vms.internal -{% endraw %} + {% endhighlight %} This demonstrates what would have been 20 separate entries (indicating scheduling failure) collapsed/compressed down to 5 entries. diff --git a/v1.1/docs/design/expansion.md b/v1.1/docs/design/expansion.md index e6ed8b1332..4506e86674 100644 --- a/v1.1/docs/design/expansion.md +++ b/v1.1/docs/design/expansion.md @@ -1,10 +1,6 @@ --- title: "Variable expansion in pod command, args, and env" --- - - -# Variable expansion in pod command, args, and env - ## Abstract A proposal for the expansion of environment variables using a simple `$(var)` syntax. @@ -170,7 +166,7 @@ which may not be bound because the service that provides them does not yet exist mapping function that uses a list of `map[string]string` like: {% highlight go %} -{% raw %} + func MakeMappingFunc(maps ...map[string]string) func(string) string { return func(input string) string { for _, context := range maps { @@ -201,7 +197,7 @@ mapping := MakeMappingFunc(containerEnv) // default variables not found in serviceEnv mappingWithDefaults := MakeMappingFunc(serviceEnv, containerEnv) -{% endraw %} + {% endhighlight %} ### Implementation changes @@ -224,7 +220,7 @@ must be able to create an event. In order to facilitate this, we should create the `api/client/record` package which is similar to `EventRecorder`, but scoped to a single object: {% highlight go %} -{% raw %} + // ObjectEventRecorder knows how to record events about a single object. type ObjectEventRecorder interface { // Event constructs an event from the given information and puts it in the queue for sending. @@ -242,14 +238,14 @@ type ObjectEventRecorder interface { // PastEventf is just like Eventf, but with an option to specify the event's 'timestamp' field. PastEventf(timestamp unversioned.Time, reason, messageFmt string, args ...interface{}) } -{% endraw %} + {% endhighlight %} There should also be a function that can construct an `ObjectEventRecorder` from a `runtime.Object` and an `EventRecorder`: {% highlight go %} -{% raw %} + type objectRecorderImpl struct { object runtime.Object recorder EventRecorder @@ -262,7 +258,7 @@ func (r *objectRecorderImpl) Event(reason, message string) { func ObjectEventRecorderFor(object runtime.Object, recorder EventRecorder) ObjectEventRecorder { return &objectRecorderImpl{object, recorder} } -{% endraw %} + {% endhighlight %} #### Expansion package @@ -270,7 +266,7 @@ func ObjectEventRecorderFor(object runtime.Object, recorder EventRecorder) Objec The expansion package should provide two methods: {% highlight go %} -{% raw %} + // MappingFuncFor returns a mapping function for use with Expand that // implements the expansion semantics defined in the expansion spec; it // returns the input string wrapped in the expansion syntax if no mapping @@ -286,7 +282,7 @@ func MappingFuncFor(recorder record.ObjectEventRecorder, context ...map[string]s func Expand(input string, mapping func(string) string) string { // ... } -{% endraw %} + {% endhighlight %} #### Kubelet changes @@ -361,7 +357,7 @@ No other variables are defined. Notice the `$(var)` syntax. {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: @@ -375,13 +371,13 @@ spec: - name: PUBLIC_URL value: "http://$(GITSERVER_SERVICE_HOST):$(GITSERVER_SERVICE_PORT)" restartPolicy: Never -{% endraw %} + {% endhighlight %} #### In a pod: building a URL using downward API {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: @@ -399,7 +395,7 @@ spec: - name: PUBLIC_URL value: "http://gitserver.$(POD_NAMESPACE):$(SERVICE_PORT)" restartPolicy: Never -{% endraw %} + {% endhighlight %} diff --git a/v1.1/docs/design/extending-api.md b/v1.1/docs/design/extending-api.md index 721af53f49..a179ffeb95 100644 --- a/v1.1/docs/design/extending-api.md +++ b/v1.1/docs/design/extending-api.md @@ -1,10 +1,6 @@ --- title: "Adding custom resources to the Kubernetes API server" --- - - -# Adding custom resources to the Kubernetes API server - This document describes the design for implementing the storage of custom API types in the Kubernetes API Server. @@ -60,7 +56,7 @@ using '-' instead of capitalization ('camel-case'), with the first character bei capitalized. In pseudo code: {% highlight go %} -{% raw %} + var result string for ix := range kindName { if isCapital(kindName[ix]) { @@ -68,7 +64,7 @@ for ix := range kindName { } result = append(result, toLowerCase(kindName[ix]) } -{% endraw %} + {% endhighlight %} As a concrete example, the resource named `camel-case-kind.example.com` defines resources of Kind `CamelCaseKind`, in @@ -86,7 +82,7 @@ deleting a namespace, deletes all third party resources in that namespace. For example, if a user creates: {% highlight yaml %} -{% raw %} + metadata: name: cron-tab.example.com apiVersion: extensions/v1beta1 @@ -95,7 +91,7 @@ description: "A specification of a Pod to run on a cron style schedule" versions: - name: stable/v1 - name: experimental/v2 -{% endraw %} + {% endhighlight %} Then the API server will program in two new RESTful resource paths: @@ -106,7 +102,7 @@ Then the API server will program in two new RESTful resource paths: Now that this schema has been created, a user can `POST`: {% highlight json %} -{% raw %} + { "metadata": { "name": "my-new-cron-object" @@ -116,7 +112,7 @@ Now that this schema has been created, a user can `POST`: "cronSpec": "* * * * /5", "image": "my-awesome-chron-image" } -{% endraw %} + {% endhighlight %} to: `/third-party/example.com/stable/v1/namespaces/default/crontabs/my-new-cron-object` @@ -124,9 +120,9 @@ to: `/third-party/example.com/stable/v1/namespaces/default/crontabs/my-new-cron- and the corresponding data will be stored into etcd by the APIServer, so that when the user issues: ``` -{% raw %} + GET /third-party/example.com/stable/v1/namespaces/default/crontabs/my-new-cron-object` -{% endraw %} + ``` And when they do that, they will get back the same data, but with additional Kubernetes metadata @@ -135,15 +131,15 @@ And when they do that, they will get back the same data, but with additional Kub Likewise, to list all resources, a user can issue: ``` -{% raw %} + GET /third-party/example.com/stable/v1/namespaces/default/crontabs -{% endraw %} + ``` and get back: {% highlight json %} -{% raw %} + { "apiVersion": "example.com/stable/v1", "kind": "CronTabList", @@ -159,7 +155,7 @@ and get back: } ] } -{% endraw %} + {% endhighlight %} Because all objects are expected to contain standard Kubernetes metadata fields, these @@ -191,17 +187,17 @@ Each custom object stored by the API server needs a custom key in storage, this Given the definitions above, the key for a specific third-party object is: ``` -{% raw %} + ${standard-k8s-prefix}/third-party-resources/${third-party-resource-namespace}/${third-party-resource-name}/${resource-namespace}/${resource-name} -{% endraw %} + ``` Thus, listing a third-party resource can be achieved by listing the directory: ``` -{% raw %} + ${standard-k8s-prefix}/third-party-resources/${third-party-resource-namespace}/${third-party-resource-name}/${resource-namespace}/ -{% endraw %} + ``` diff --git a/v1.1/docs/design/horizontal-pod-autoscaler.md b/v1.1/docs/design/horizontal-pod-autoscaler.md index 13b7200bf1..42c616aefb 100644 --- a/v1.1/docs/design/horizontal-pod-autoscaler.md +++ b/v1.1/docs/design/horizontal-pod-autoscaler.md @@ -1,10 +1,6 @@ --- title: "Horizontal Pod Autoscaling" --- - - -# Horizontal Pod Autoscaling - ## Preface This document briefly describes the design of the horizontal autoscaler for pods. @@ -42,7 +38,7 @@ Scale subresource is in API for replication controller or deployment under the f It has the following structure: {% highlight go %} -{% raw %} + // represents a scaling request for a resource. type Scale struct { unversioned.TypeMeta @@ -69,7 +65,7 @@ type ScaleStatus struct { // label query over pods that should match the replicas count. Selector map[string]string `json:"selector,omitempty"` } -{% endraw %} + {% endhighlight %} Writing to `ScaleSpec.Replicas` resizes the replication controller/deployment associated with @@ -86,7 +82,7 @@ In Kubernetes version 1.1, we are introducing HorizontalPodAutoscaler object. It It has the following structure: {% highlight go %} -{% raw %} + // configuration of a horizontal pod autoscaler. type HorizontalPodAutoscaler struct { unversioned.TypeMeta @@ -139,7 +135,7 @@ type HorizontalPodAutoscalerStatus struct { // e.g. 70 means that an average pod is using now 70% of its requested CPU. CurrentCPUUtilizationPercentage *int } -{% endraw %} + {% endhighlight %} `ScaleRef` is a reference to the Scale subresource. @@ -147,7 +143,7 @@ type HorizontalPodAutoscalerStatus struct { We are also introducing HorizontalPodAutoscalerList object to enable listing all autoscalers in a namespace: {% highlight go %} -{% raw %} + // list of horizontal pod autoscaler objects. type HorizontalPodAutoscalerList struct { unversioned.TypeMeta @@ -156,7 +152,7 @@ type HorizontalPodAutoscalerList struct { // list of horizontal pod autoscaler objects. Items []HorizontalPodAutoscaler } -{% endraw %} + {% endhighlight %} ## Autoscaling Algorithm @@ -178,9 +174,9 @@ In future, there will be API on master for this purpose The target number of pods is calculated from the following formula: ``` -{% raw %} + TargetNumOfPods = ceil(sum(CurrentPodsCPUUtilization) / Target) -{% endraw %} + ``` Starting and stopping pods may introduce noise to the metric (for instance, starting may temporarily increase CPU). diff --git a/v1.1/docs/design/identifiers.md b/v1.1/docs/design/identifiers.md index a79fcda106..9936c649b4 100644 --- a/v1.1/docs/design/identifiers.md +++ b/v1.1/docs/design/identifiers.md @@ -1,10 +1,6 @@ --- title: "Identifiers and Names in Kubernetes" --- - - -# Identifiers and Names in Kubernetes - A summarization of the goals and recommendations for identifiers in Kubernetes. Described in [GitHub issue #199](http://issue.k8s.io/199). diff --git a/v1.1/docs/design/index.md b/v1.1/docs/design/index.md index 8cfa3289ec..69a6362161 100644 --- a/v1.1/docs/design/index.md +++ b/v1.1/docs/design/index.md @@ -1,10 +1,6 @@ --- title: "Kubernetes Design Overview" --- - - -# Kubernetes Design Overview - Kubernetes is a system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications. Kubernetes establishes robust declarative primitives for maintaining the desired state requested by the user. We see these primitives as the main value added by Kubernetes. Self-healing mechanisms, such as auto-restarting, re-scheduling, and replicating containers require active controllers, not just imperative orchestration. @@ -15,11 +11,11 @@ Kubernetes enables users to ask a cluster to run a set of containers. The system Kubernetes is intended to run on a number of cloud providers, as well as on physical hosts. -A single Kubernetes cluster is not intended to span multiple availability zones. Instead, we recommend building a higher-level layer to replicate complete deployments of highly available applications across multiple zones (see [the multi-cluster doc](../admin/multi-cluster.html) and [cluster federation proposal](../proposals/federation.html) for more details). +A single Kubernetes cluster is not intended to span multiple availability zones. Instead, we recommend building a higher-level layer to replicate complete deployments of highly available applications across multiple zones (see [the multi-cluster doc](../admin/multi-cluster) and [cluster federation proposal](../proposals/federation) for more details). Finally, Kubernetes aspires to be an extensible, pluggable, building-block OSS platform and toolkit. Therefore, architecturally, we want Kubernetes to be built as a collection of pluggable components and layers, with the ability to use alternative schedulers, controllers, storage systems, and distribution mechanisms, and we're evolving its current code in that direction. Furthermore, we want others to be able to extend Kubernetes functionality, such as with higher-level PaaS functionality or multi-cluster layers, without modification of core Kubernetes source. Therefore, its API isn't just (or even necessarily mainly) targeted at end users, but at tool and extension developers. Its APIs are intended to serve as the foundation for an open ecosystem of tools, automation systems, and higher-level API layers. Consequently, there are no "internal" inter-component APIs. All APIs are visible and available, including the APIs used by the scheduler, the node controller, the replication-controller manager, Kubelet's API, etc. There's no glass to break -- in order to handle more complex use cases, one can just access the lower-level APIs in a fully transparent, composable manner. -For more about the Kubernetes architecture, see [architecture](architecture.html). +For more about the Kubernetes architecture, see [architecture](architecture). diff --git a/v1.1/docs/design/namespaces.md b/v1.1/docs/design/namespaces.md index 3390ed0086..3fead4dea7 100644 --- a/v1.1/docs/design/namespaces.md +++ b/v1.1/docs/design/namespaces.md @@ -1,10 +1,6 @@ --- title: "Namespaces" --- - - -# Namespaces - ## Abstract A Namespace is a mechanism to partition resources created by users into @@ -47,7 +43,7 @@ The Namespace provides a unique scope for: A *Namespace* defines a logically named group for multiple *Kind*s of resources. {% highlight go %} -{% raw %} + type Namespace struct { TypeMeta `json:",inline"` ObjectMeta `json:"metadata,omitempty"` @@ -55,7 +51,7 @@ type Namespace struct { Spec NamespaceSpec `json:"spec,omitempty"` Status NamespaceStatus `json:"status,omitempty"` } -{% endraw %} + {% endhighlight %} A *Namespace* name is a DNS compatible label. @@ -79,7 +75,7 @@ distinguish distinct entities, and reference particular entities across operatio A *Namespace* provides an authorization scope for accessing content associated with the *Namespace*. -See [Authorization plugins](../admin/authorization.html) +See [Authorization plugins](../admin/authorization) ### Limit Resource Consumption @@ -88,19 +84,19 @@ A *Namespace* provides a scope to limit resource consumption. A *LimitRange* defines min/max constraints on the amount of resources a single entity can consume in a *Namespace*. -See [Admission control: Limit Range](admission_control_limit_range.html) +See [Admission control: Limit Range](admission_control_limit_range) A *ResourceQuota* tracks aggregate usage of resources in the *Namespace* and allows cluster operators to define *Hard* resource usage limits that a *Namespace* may consume. -See [Admission control: Resource Quota](admission_control_resource_quota.html) +See [Admission control: Resource Quota](admission_control_resource_quota) ### Finalizers Upon creation of a *Namespace*, the creator may provide a list of *Finalizer* objects. {% highlight go %} -{% raw %} + type FinalizerName string // These are internal finalizers to Kubernetes, must be qualified name unless defined here @@ -113,7 +109,7 @@ type NamespaceSpec struct { // Finalizers is an opaque list of values that must be empty to permanently remove object from storage Finalizers []FinalizerName } -{% endraw %} + {% endhighlight %} A *FinalizerName* is a qualified name. @@ -131,7 +127,7 @@ set by default. A *Namespace* may exist in the following phases. {% highlight go %} -{% raw %} + type NamespacePhase string const( NamespaceActive NamespacePhase = "Active" @@ -142,7 +138,7 @@ type NamespaceStatus struct { ... Phase NamespacePhase } -{% endraw %} + {% endhighlight %} A *Namespace* is in the **Active** phase if it does not have a *ObjectMeta.DeletionTimestamp*. @@ -241,7 +237,7 @@ to take part in Namespace termination. OpenShift creates a Namespace in Kubernetes {% highlight json %} -{% raw %} + { "apiVersion":"v1", "kind": "Namespace", @@ -258,7 +254,7 @@ OpenShift creates a Namespace in Kubernetes "phase": "Active" } } -{% endraw %} + {% endhighlight %} OpenShift then goes and creates a set of resources (pods, services, etc) associated @@ -268,7 +264,7 @@ own storage associated with the "development" namespace unknown to Kubernetes. User deletes the Namespace in Kubernetes, and Namespace now has following state: {% highlight json %} -{% raw %} + { "apiVersion":"v1", "kind": "Namespace", @@ -286,7 +282,7 @@ User deletes the Namespace in Kubernetes, and Namespace now has following state: "phase": "Terminating" } } -{% endraw %} + {% endhighlight %} The Kubernetes *namespace controller* observes the namespace has a *deletionTimestamp* @@ -295,7 +291,7 @@ success, it executes a *finalize* action that modifies the *Namespace* by removing *kubernetes* from the list of finalizers: {% highlight json %} -{% raw %} + { "apiVersion":"v1", "kind": "Namespace", @@ -313,7 +309,7 @@ removing *kubernetes* from the list of finalizers: "phase": "Terminating" } } -{% endraw %} + {% endhighlight %} OpenShift Origin has its own *namespace controller* that is observing cluster state, and @@ -325,7 +321,7 @@ from the list of finalizers. This results in the following state: {% highlight json %} -{% raw %} + { "apiVersion":"v1", "kind": "Namespace", @@ -343,7 +339,7 @@ This results in the following state: "phase": "Terminating" } } -{% endraw %} + {% endhighlight %} At this point, the Kubernetes *namespace controller* in its sync loop will see that the namespace diff --git a/v1.1/docs/design/networking.md b/v1.1/docs/design/networking.md index 559885e356..ae4c04265f 100644 --- a/v1.1/docs/design/networking.md +++ b/v1.1/docs/design/networking.md @@ -1,10 +1,6 @@ --- title: "Networking" --- - - -# Networking - There are 4 distinct networking problems to solve: 1. Highly-coupled container-to-container communications @@ -40,7 +36,7 @@ among other problems. ## Container to container All containers within a pod behave as if they are on the same host with regard -to networking. They can all reach each other’s ports on localhost. This offers +to networking. They can all reach each other's ports on localhost. This offers simplicity (static ports know a priori), security (ports bound to localhost are visible within the pod but never outside it), and performance. This also reduces friction for applications moving from the world of uncontainerized apps @@ -104,14 +100,14 @@ differentiate it from `docker0`) is set up outside of Docker proper. Example of GCE's advanced routing rules: {% highlight sh %} -{% raw %} + gcloud compute routes add "${MINION_NAMES[$i]}" \ --project "${PROJECT}" \ --destination-range "${MINION_IP_RANGES[$i]}" \ --network "${NETWORK}" \ --next-hop-instance "${MINION_NAMES[$i]}" \ --next-hop-instance-zone "${ZONE}" & -{% endraw %} + {% endhighlight %} GCE itself does not know anything about these IPs, though. This means that when @@ -122,7 +118,7 @@ a pod tries to egress beyond GCE's project the packets must be SNAT'ed With the primary aim of providing IP-per-pod-model, other implementations exist to serve the purpose outside of GCE. - - [OpenVSwitch with GRE/VxLAN](../admin/ovs-networking.html) + - [OpenVSwitch with GRE/VxLAN](../admin/ovs-networking) - [Flannel](https://github.com/coreos/flannel#flannel) - [L2 networks](http://blog.oddbit.com/2014/08/11/four-ways-to-connect-a-docker/) ("With Linux Bridge devices" section) @@ -133,7 +129,7 @@ to serve the purpose outside of GCE. ## Pod to service -The [service](../user-guide/services.html) abstraction provides a way to group pods under a +The [service](../user-guide/services) abstraction provides a way to group pods under a common access policy (e.g. load-balanced). The implementation of this creates a virtual IP which clients can access and which is transparently proxied to the pods in a Service. Each node runs a kube-proxy process which programs diff --git a/v1.1/docs/design/persistent-storage.md b/v1.1/docs/design/persistent-storage.md index cd2216c692..8c9bc74800 100644 --- a/v1.1/docs/design/persistent-storage.md +++ b/v1.1/docs/design/persistent-storage.md @@ -1,10 +1,6 @@ --- title: "Persistent Storage" --- - - -# Persistent Storage - This document proposes a model for managing persistent, cluster-scoped storage for applications requiring long lived data. ### tl;dr @@ -100,7 +96,7 @@ Events that communicate the state of a mounted volume are left to the volume plu An administrator provisions storage by posting PVs to the API. Various way to automate this task can be scripted. Dynamic provisioning is a future feature that can maintain levels of PVs. {% highlight yaml %} -{% raw %} + POST: kind: PersistentVolume @@ -113,16 +109,16 @@ spec: persistentDisk: pdName: "abc123" fsType: "ext4" -{% endraw %} + {% endhighlight %} {% highlight console %} -{% raw %} + $ kubectl get pv NAME LABELS CAPACITY ACCESSMODES STATUS CLAIM REASON pv0001 map[] 10737418240 RWO Pending -{% endraw %} + {% endhighlight %} #### Users request storage @@ -132,7 +128,7 @@ A user requests storage by posting a PVC to the API. Their request contains the The user must be within a namespace to create PVCs. {% highlight yaml %} -{% raw %} + POST: kind: PersistentVolumeClaim @@ -145,16 +141,16 @@ spec: resources: requests: storage: 3 -{% endraw %} + {% endhighlight %} {% highlight console %} -{% raw %} + $ kubectl get pvc NAME LABELS STATUS VOLUME myclaim-1 map[] pending -{% endraw %} + {% endhighlight %} @@ -163,7 +159,7 @@ myclaim-1 map[] pending The ```PersistentVolumeClaimBinder``` attempts to find an available volume that most closely matches the user's request. If one exists, they are bound by putting a reference on the PV to the PVC. Requests can go unfulfilled if a suitable match is not found. {% highlight console %} -{% raw %} + $ kubectl get pv NAME LABELS CAPACITY ACCESSMODES STATUS CLAIM REASON @@ -174,7 +170,7 @@ kubectl get pvc NAME LABELS STATUS VOLUME myclaim-1 map[] Bound b16e91d6-c0ef-11e4-8be4-80e6500a981e -{% endraw %} + {% endhighlight %} #### Claim usage @@ -184,7 +180,7 @@ The claim holder can use their claim as a volume. The ```PersistentVolumeClaimV The claim holder owns the claim and its data for as long as the claim exists. The pod using the claim can be deleted, but the claim remains in the user's namespace. It can be used again and again by many pods. {% highlight yaml %} -{% raw %} + POST: kind: Pod @@ -205,7 +201,7 @@ spec: accessMode: ReadWriteOnce claimRef: name: myclaim-1 -{% endraw %} + {% endhighlight %} #### Releasing a claim and Recycling a volume @@ -213,9 +209,9 @@ spec: When a claim holder is finished with their data, they can delete their claim. {% highlight console %} -{% raw %} + $ kubectl delete pvc myclaim-1 -{% endraw %} + {% endhighlight %} The ```PersistentVolumeClaimBinder``` will reconcile this by removing the claim reference from the PV and change the PVs status to 'Released'. diff --git a/v1.1/docs/design/principles.md b/v1.1/docs/design/principles.md index 72babefcf6..a8628c82c4 100644 --- a/v1.1/docs/design/principles.md +++ b/v1.1/docs/design/principles.md @@ -1,15 +1,11 @@ --- title: "Design Principles" --- - - -# Design Principles - Principles to follow when extending Kubernetes. ## API -See also the [API conventions](../devel/api-conventions.html). +See also the [API conventions](../devel/api-conventions). * All APIs should be declarative. * API objects should be complementary and composable, not opaque wrappers. diff --git a/v1.1/docs/design/resources.md b/v1.1/docs/design/resources.md index 681b18ec7e..e1a96c1db7 100644 --- a/v1.1/docs/design/resources.md +++ b/v1.1/docs/design/resources.md @@ -3,7 +3,7 @@ title: "The Kubernetes resource model" --- **Note: this is a design doc, which describes features that have not been completely implemented. -User documentation of the current state is [here](../user-guide/compute-resources.html). The tracking issue for +User documentation of the current state is [here](../user-guide/compute-resources). The tracking issue for implementation of this model is [#168](http://issue.k8s.io/168). Currently, both limits and requests of memory and cpu on containers (not pods) are supported. "memory" is in bytes and "cpu" is in @@ -62,12 +62,12 @@ Both users and a number of system components, such as schedulers, (horizontal) a Resource requirements for a container or pod should have the following form: {% highlight yaml %} -{% raw %} + resourceRequirementSpec: [ request: [ cpu: 2.5, memory: "40Mi" ], limit: [ cpu: 4.0, memory: "99Mi" ], ] -{% endraw %} + {% endhighlight %} Where: @@ -78,11 +78,11 @@ Where: Total capacity for a node should have a similar structure: {% highlight yaml %} -{% raw %} + resourceCapacitySpec: [ total: [ cpu: 12, memory: "128Gi" ] ] -{% endraw %} + {% endhighlight %} Where: @@ -113,7 +113,7 @@ The following resource types are predefined ("reserved") by Kubernetes in the `k * [future] `schedulingLatency`: as per lmctfy * [future] `cpuConversionFactor`: property of a node: the speed of a CPU core on the node's processor divided by the speed of the canonical Kubernetes CPU (a floating point value; default = 1.0). -To reduce performance portability problems for pods, and to avoid worse-case provisioning behavior, the units of CPU will be normalized to a canonical "Kubernetes Compute Unit" (KCU, pronounced ˈkoÍ?okoÍžo), which will roughly be equivalent to a single CPU hyperthreaded core for some recent x86 processor. The normalization may be implementation-defined, although some reasonable defaults will be provided in the open-source Kubernetes code. +To reduce performance portability problems for pods, and to avoid worse-case provisioning behavior, the units of CPU will be normalized to a canonical "Kubernetes Compute Unit" (KCU, pronounced ˈkoÃ?okoÞo), which will roughly be equivalent to a single CPU hyperthreaded core for some recent x86 processor. The normalization may be implementation-defined, although some reasonable defaults will be provided in the open-source Kubernetes code. Note that requesting 2 KCU won't guarantee that precisely 2 physical cores will be allocated — control of aspects like this will be handled by resource _qualities_ (a future feature). @@ -135,7 +135,7 @@ rather than decimal ones: "64MiB" rather than "64MB". A resource type may have an associated read-only ResourceType structure, that contains metadata about the type. For example: {% highlight yaml %} -{% raw %} + resourceTypes: [ "kubernetes.io/memory": [ isCompressible: false, ... @@ -146,7 +146,7 @@ resourceTypes: [ ] "kubernetes.io/disk-space": [ ... ] ] -{% endraw %} + {% endhighlight %} Kubernetes will provide ResourceType metadata for its predefined types. If no resource metadata can be found for a resource type, Kubernetes will assume that it is a quantified, incompressible resource that is not specified in milli-units, and has no default value. @@ -169,24 +169,24 @@ The following are planned future extensions to the resource model, included here ## Usage data -Because resource usage and related metrics change continuously, need to be tracked over time (i.e., historically), can be characterized in a variety of ways, and are fairly voluminous, we will not include usage in core API objects, such as [Pods](../user-guide/pods.html) and Nodes, but will provide separate APIs for accessing and managing that data. See the Appendix for possible representations of usage data, but the representation we'll use is TBD. +Because resource usage and related metrics change continuously, need to be tracked over time (i.e., historically), can be characterized in a variety of ways, and are fairly voluminous, we will not include usage in core API objects, such as [Pods](../user-guide/pods) and Nodes, but will provide separate APIs for accessing and managing that data. See the Appendix for possible representations of usage data, but the representation we'll use is TBD. Singleton values for observed and predicted future usage will rapidly prove inadequate, so we will support the following structure for extended usage information: {% highlight yaml %} -{% raw %} + resourceStatus: [ usage: [ cpu: , memory: ], maxusage: [ cpu: , memory: ], predicted: [ cpu: , memory: ], ] -{% endraw %} + {% endhighlight %} where a `` or `` structure looks like this: {% highlight yaml %} -{% raw %} + { mean: # arithmetic mean max: # minimum value @@ -200,7 +200,7 @@ where a `` or `` structure looks like this: ... ] } -{% endraw %} + {% endhighlight %} All parts of this structure are optional, although we strongly encourage including quantities for 50, 90, 95, 99, 99.5, and 99.9 percentiles. _[In practice, it will be important to include additional info such as the length of the time window over which the averages are calculated, the confidence level, and information-quality metrics such as the number of dropped or discarded data points.]_ @@ -243,5 +243,3 @@ This is the amount of time a container spends accessing disk, including actuator * Units: operations per second * Compressible? yes - - diff --git a/v1.1/docs/design/secrets.md b/v1.1/docs/design/secrets.md index 3636036400..cb2ce67c20 100644 --- a/v1.1/docs/design/secrets.md +++ b/v1.1/docs/design/secrets.md @@ -1,11 +1,7 @@ --- title: "Abstract" --- - - -## Abstract - -A proposal for the distribution of [secrets](../user-guide/secrets.html) (passwords, keys, etc) to the Kubelet and to +A proposal for the distribution of [secrets](../user-guide/secrets) (passwords, keys, etc) to the Kubelet and to containers inside Kubernetes using a custom [volume](../user-guide/volumes.html#secrets) type. See the [secrets example](../user-guide/secrets/) for more information. ## Motivation @@ -75,7 +71,7 @@ service would also consume the secrets associated with the MySQL service. ### Use-Case: Secrets associated with service accounts -[Service Accounts](service_accounts.html) are proposed as a +[Service Accounts](service_accounts) are proposed as a mechanism to decouple capabilities and security contexts from individual human users. A `ServiceAccount` contains references to some number of secrets. A `Pod` can specify that it is associated with a `ServiceAccount`. Secrets should have a `Type` field to allow the Kubelet and @@ -245,7 +241,7 @@ memory overcommit on the node. #### Secret data on the node: isolation -Every pod will have a [security context](security_context.html). +Every pod will have a [security context](security_context). Secret data on the node should be isolated according to the security context of the container. The Kubelet volume plugin API will be changed so that a volume plugin receives the security context of a volume along with the volume spec. This will allow volume plugins to implement setting the @@ -257,7 +253,7 @@ Several proposals / upstream patches are notable as background for this proposal 1. [Docker vault proposal](https://github.com/docker/docker/issues/10310) 2. [Specification for image/container standardization based on volumes](https://github.com/docker/docker/issues/9277) -3. [Kubernetes service account proposal](service_accounts.html) +3. [Kubernetes service account proposal](service_accounts) 4. [Secrets proposal for docker (1)](https://github.com/docker/docker/pull/6075) 5. [Secrets proposal for docker (2)](https://github.com/docker/docker/pull/6697) @@ -277,7 +273,7 @@ in the container specification. A new resource for secrets will be added to the API: {% highlight go %} -{% raw %} + type Secret struct { TypeMeta ObjectMeta @@ -301,7 +297,7 @@ const ( ) const MaxSecretSize = 1 * 1024 * 1024 -{% endraw %} + {% endhighlight %} A Secret can declare a type in order to provide type information to system components that work @@ -327,7 +323,7 @@ A new `SecretSource` type of volume source will be added to the `VolumeSource` s API: {% highlight go %} -{% raw %} + type VolumeSource struct { // Other fields omitted @@ -338,7 +334,7 @@ type VolumeSource struct { type SecretSource struct { Target ObjectReference } -{% endraw %} + {% endhighlight %} Secret volume sources are validated to ensure that the specified object reference actually points @@ -356,14 +352,14 @@ require access to the API server to retrieve secret data and therefore the volum will have to change to expose a client interface: {% highlight go %} -{% raw %} + type Host interface { // Other methods omitted // GetKubeClient returns a client interface GetKubeClient() client.Interface } -{% endraw %} + {% endhighlight %} The secret volume plugin will be responsible for: @@ -403,7 +399,7 @@ suggested changes. All of these examples are assumed to be created in a namespa To create a pod that uses an ssh key stored as a secret, we first need to create a secret: {% highlight json %} -{% raw %} + { "kind": "Secret", "apiVersion": "v1", @@ -415,7 +411,7 @@ To create a pod that uses an ssh key stored as a secret, we first need to create "id-rsa.pub": "dmFsdWUtMQ0K" } } -{% endraw %} + {% endhighlight %} **Note:** The serialized JSON and YAML values of secret data are encoded as @@ -425,7 +421,7 @@ omitted. Now we can create a pod which references the secret with the ssh key and consumes it in a volume: {% highlight json %} -{% raw %} + { "kind": "Pod", "apiVersion": "v1", @@ -459,7 +455,7 @@ Now we can create a pod which references the secret with the ssh key and consume ] } } -{% endraw %} + {% endhighlight %} When the container's command runs, the pieces of the key will be available in: @@ -478,7 +474,7 @@ credentials. The secrets: {% highlight json %} -{% raw %} + { "apiVersion": "v1", "kind": "List", @@ -506,13 +502,13 @@ The secrets: } }] } -{% endraw %} + {% endhighlight %} The pods: {% highlight json %} -{% raw %} + { "apiVersion": "v1", "kind": "List", @@ -584,7 +580,7 @@ The pods: } }] } -{% endraw %} + {% endhighlight %} The specs for the two pods differ only in the value of the object referred to by the secret volume diff --git a/v1.1/docs/design/security.md b/v1.1/docs/design/security.md index 4a891d21da..686d0d7254 100644 --- a/v1.1/docs/design/security.md +++ b/v1.1/docs/design/security.md @@ -1,10 +1,6 @@ --- title: "Security in Kubernetes" --- - - -# Security in Kubernetes - Kubernetes should define a reasonable set of security best practices that allows processes to be isolated from each other, from the cluster infrastructure, and which preserves important boundaries between those who manage the cluster, and those who use the cluster. While Kubernetes today is not primarily a multi-tenant system, the long term evolution of Kubernetes will increasingly rely on proper boundaries between users and administrators. The code running on the cluster must be appropriately isolated and secured to prevent malicious parties from affecting the entire cluster. @@ -68,14 +64,14 @@ Automated process users fall into the following categories: A pod runs in a *security context* under a *service account* that is defined by an administrator or project administrator, and the *secrets* a pod has access to is limited by that *service account*. -1. The API should authenticate and authorize user actions [authn and authz](access.html) +1. The API should authenticate and authorize user actions [authn and authz](access) 2. All infrastructure components (kubelets, kube-proxies, controllers, scheduler) should have an infrastructure user that they can authenticate with and be authorized to perform only the functions they require against the API. 3. Most infrastructure components should use the API as a way of exchanging data and changing the system, and only the API should have access to the underlying data store (etcd) -4. When containers run on the cluster and need to talk to other containers or the API server, they should be identified and authorized clearly as an autonomous process via a [service account](service_accounts.html) +4. When containers run on the cluster and need to talk to other containers or the API server, they should be identified and authorized clearly as an autonomous process via a [service account](service_accounts) 1. If the user who started a long-lived process is removed from access to the cluster, the process should be able to continue without interruption 2. If the user who started processes are removed from the cluster, administrators may wish to terminate their processes in bulk 3. When containers run with a service account, the user that created / triggered the service account behavior must be associated with the container's action -5. When container processes run on the cluster, they should run in a [security context](security_context.html) that isolates those processes via Linux user security, user namespaces, and permissions. +5. When container processes run on the cluster, they should run in a [security context](security_context) that isolates those processes via Linux user security, user namespaces, and permissions. 1. Administrators should be able to configure the cluster to automatically confine all container processes as a non-root, randomly assigned UID 2. Administrators should be able to ensure that container processes within the same namespace are all assigned the same unix user UID 3. Administrators should be able to limit which developers and project administrators have access to higher privilege actions @@ -84,7 +80,7 @@ A pod runs in a *security context* under a *service account* that is defined by 6. Developers may need to ensure their images work within higher security requirements specified by administrators 7. When available, Linux kernel user namespaces can be used to ensure 5.2 and 5.4 are met. 8. When application developers want to share filesystem data via distributed filesystems, the Unix user ids on those filesystems must be consistent across different container processes -6. Developers should be able to define [secrets](secrets.html) that are automatically added to the containers when pods are run +6. Developers should be able to define [secrets](secrets) that are automatically added to the containers when pods are run 1. Secrets are files injected into the container whose values should not be displayed within a pod. Examples: 1. An SSH private key for git cloning remote data 2. A client certificate for accessing a remote system @@ -98,11 +94,11 @@ A pod runs in a *security context* under a *service account* that is defined by ### Related design discussion -* [Authorization and authentication](access.html) +* [Authorization and authentication](access) * [Secret distribution via files](http://pr.k8s.io/2030) * [Docker secrets](https://github.com/docker/docker/pull/6697) * [Docker vault](https://github.com/docker/docker/issues/10310) -* [Service Accounts:](service_accounts.html) +* [Service Accounts:](service_accounts) * [Secret volumes](http://pr.k8s.io/4126) ## Specific Design Points diff --git a/v1.1/docs/design/security_context.md b/v1.1/docs/design/security_context.md index 8a10a93cd9..5613ca676f 100644 --- a/v1.1/docs/design/security_context.md +++ b/v1.1/docs/design/security_context.md @@ -1,13 +1,9 @@ --- title: "Security Contexts" --- - - -# Security Contexts - ## Abstract -A security context is a set of constraints that are applied to a container in order to achieve the following goals (from [security design](security.html)): +A security context is a set of constraints that are applied to a container in order to achieve the following goals (from [security design](security)): 1. Ensure a clear isolation between container and the underlying host it runs on 2. Limit the ability of the container to negatively impact the infrastructure or other containers @@ -41,7 +37,7 @@ Processes in pods will need to have consistent UID/GID/SELinux category labels i * The concept of a security context should not be tied to a particular security mechanism or platform (ie. SELinux, AppArmor) * Applying a different security context to a scope (namespace or pod) requires a solution such as the one proposed for - [service accounts](service_accounts.html). + [service accounts](service_accounts). ## Use Cases @@ -92,7 +88,7 @@ It is recommended that this design be implemented in two phases: The Kubelet will have an interface that points to a `SecurityContextProvider`. The `SecurityContextProvider` is invoked before creating and running a given container: {% highlight go %} -{% raw %} + type SecurityContextProvider interface { // ModifyContainerConfig is called before the Docker createContainer call. // The security context provider can make changes to the Config with which @@ -108,7 +104,7 @@ type SecurityContextProvider interface { // with a security context. ModifyHostConfig(pod *api.Pod, container *api.Container, hostConfig *docker.HostConfig) } -{% endraw %} + {% endhighlight %} If the value of the SecurityContextProvider field on the Kubelet is nil, the kubelet will create and run the container as it does today. @@ -119,7 +115,7 @@ A security context resides on the container and represents the runtime parameter be used to create and run the container via container APIs. Following is an example of an initial implementation: {% highlight go %} -{% raw %} + type Container struct { ... other fields omitted ... // Optional: SecurityContext defines the security options the pod should be run with @@ -159,7 +155,7 @@ type SELinuxOptions struct { // SELinux level label. Level string } -{% endraw %} + {% endhighlight %} ### Admission diff --git a/v1.1/docs/design/service_accounts.md b/v1.1/docs/design/service_accounts.md index bf26ea6e72..edfc33332a 100644 --- a/v1.1/docs/design/service_accounts.md +++ b/v1.1/docs/design/service_accounts.md @@ -1,10 +1,6 @@ --- title: "Service Accounts" --- - - -# Service Accounts - ## Motivation Processes in Pods may need to call the Kubernetes API. For example: @@ -26,10 +22,10 @@ They also may interact with services other than the Kubernetes API, such as: A service account binds together several things: - a *name*, understood by users, and perhaps by peripheral systems, for an identity - - a *principal* that can be authenticated and [authorized](../admin/authorization.html) - - a [security context](security_context.html), which defines the Linux Capabilities, User IDs, Groups IDs, and other + - a *principal* that can be authenticated and [authorized](../admin/authorization) + - a [security context](security_context), which defines the Linux Capabilities, User IDs, Groups IDs, and other capabilities and controls on interaction with the file system and OS. - - a set of [secrets](secrets.html), which a container may use to + - a set of [secrets](secrets), which a container may use to access various networked resources. ## Design Discussion @@ -37,7 +33,7 @@ A service account binds together several things: A new object Kind is added: {% highlight go %} -{% raw %} + type ServiceAccount struct { TypeMeta `json:",inline" yaml:",inline"` ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"` @@ -46,7 +42,7 @@ type ServiceAccount struct { securityContext ObjectReference // (reference to a securityContext object) secrets []ObjectReference // (references to secret objects } -{% endraw %} + {% endhighlight %} The name ServiceAccount is chosen because it is widely used already (e.g. by Kerberos and LDAP) diff --git a/v1.1/docs/design/simple-rolling-update.md b/v1.1/docs/design/simple-rolling-update.md index 6c166f332e..a1b120ef66 100644 --- a/v1.1/docs/design/simple-rolling-update.md +++ b/v1.1/docs/design/simple-rolling-update.md @@ -1,11 +1,7 @@ --- title: "Simple rolling update" --- - - -## Simple rolling update - -This is a lightweight design document for simple [rolling update](../user-guide/kubectl/kubectl_rolling-update.html) in `kubectl`. +This is a lightweight design document for simple [rolling update](../user-guide/kubectl/kubectl_rolling-update) in `kubectl`. Complete execution flow can be found [here](#execution-details). See the [example of rolling update](../user-guide/update-demo/) for more information. @@ -35,9 +31,9 @@ To facilitate recovery in the case of a crash of the updating process itself, we Recovery is achieved by issuing the same command again: {% highlight sh %} -{% raw %} + kubectl rolling-update foo [foo-v2] --image=myimage:v2 -{% endraw %} + {% endhighlight %} Whenever the rolling update command executes, the kubectl client looks for replication controllers called `foo` and `foo-next`, if they exist, an attempt is diff --git a/v1.1/docs/design/versioning.md b/v1.1/docs/design/versioning.md index 10038dcfbc..0c2657fb59 100644 --- a/v1.1/docs/design/versioning.md +++ b/v1.1/docs/design/versioning.md @@ -1,10 +1,6 @@ --- title: "Kubernetes API and Release Versioning" --- - - -# Kubernetes API and Release Versioning - Legend: * **Kube <major>.<minor>.<patch>** refers to the version of Kubernetes that is released. This versions all components: apiserver, kubelet, kubectl, etc. diff --git a/v1.1/docs/devel/README.md b/v1.1/docs/devel/README.md index ae73742b50..564ce00a81 100644 --- a/v1.1/docs/devel/README.md +++ b/v1.1/docs/devel/README.md @@ -1,47 +1,43 @@ --- title: "Kubernetes Developer Guide" --- - - -# Kubernetes Developer Guide - The developer guide is for anyone wanting to either write code which directly accesses the Kubernetes API, or to contribute directly to the Kubernetes project. -It assumes some familiarity with concepts in the [User Guide](../user-guide/README.html) and the [Cluster Admin -Guide](../admin/README.html). +It assumes some familiarity with concepts in the [User Guide](../user-guide/README) and the [Cluster Admin +Guide](../admin/README). ## The process of developing and contributing code to the Kubernetes project -* **On Collaborative Development** ([collab.md](collab.html)): Info on pull requests and code reviews. +* **On Collaborative Development** ([collab.md](collab)): Info on pull requests and code reviews. -* **GitHub Issues** ([issues.md](issues.html)): How incoming issues are reviewed and prioritized. +* **GitHub Issues** ([issues.md](issues)): How incoming issues are reviewed and prioritized. -* **Pull Request Process** ([pull-requests.md](pull-requests.html)): When and why pull requests are closed. +* **Pull Request Process** ([pull-requests.md](pull-requests)): When and why pull requests are closed. -* **Faster PR reviews** ([faster_reviews.md](faster_reviews.html)): How to get faster PR reviews. +* **Faster PR reviews** ([faster_reviews.md](faster_reviews)): How to get faster PR reviews. -* **Getting Recent Builds** ([getting-builds.md](getting-builds.html)): How to get recent builds including the latest builds that pass CI. +* **Getting Recent Builds** ([getting-builds.md](getting-builds)): How to get recent builds including the latest builds that pass CI. -* **Automated Tools** ([automation.md](automation.html)): Descriptions of the automation that is running on our github repository. +* **Automated Tools** ([automation.md](automation)): Descriptions of the automation that is running on our github repository. ## Setting up your dev environment, coding, and debugging -* **Development Guide** ([development.md](development.html)): Setting up your development environment. +* **Development Guide** ([development.md](development)): Setting up your development environment. -* **Hunting flaky tests** ([flaky-tests.md](flaky-tests.html)): We have a goal of 99.9% flake free tests. +* **Hunting flaky tests** ([flaky-tests.md](flaky-tests)): We have a goal of 99.9% flake free tests. Here's how to run your tests many times. -* **Logging Conventions** ([logging.md](logging.html)]: Glog levels. +* **Logging Conventions** ([logging.md](logging)]: Glog levels. -* **Profiling Kubernetes** ([profiling.md](profiling.html)): How to plug in go pprof profiler to Kubernetes. +* **Profiling Kubernetes** ([profiling.md](profiling)): How to plug in go pprof profiler to Kubernetes. * **Instrumenting Kubernetes with a new metric** - ([instrumentation.md](instrumentation.html)): How to add a new metrics to the + ([instrumentation.md](instrumentation)): How to add a new metrics to the Kubernetes code base. -* **Coding Conventions** ([coding-conventions.md](coding-conventions.html)): +* **Coding Conventions** ([coding-conventions.md](coding-conventions)): Coding style advice for contributors. @@ -49,33 +45,33 @@ Guide](../admin/README.html). * API objects are explained at [http://kubernetes.io/third_party/swagger-ui/](http://kubernetes.io/third_party/swagger-ui/). -* **Annotations** ([docs/user-guide/annotations.md](../user-guide/annotations.html)): are for attaching arbitrary non-identifying metadata to objects. +* **Annotations** ([docs/user-guide/annotations.md](../user-guide/annotations)): are for attaching arbitrary non-identifying metadata to objects. Programs that automate Kubernetes objects may use annotations to store small amounts of their state. -* **API Conventions** ([api-conventions.md](api-conventions.html)): +* **API Conventions** ([api-conventions.md](api-conventions)): Defining the verbs and resources used in the Kubernetes API. -* **API Client Libraries** ([client-libraries.md](client-libraries.html)): +* **API Client Libraries** ([client-libraries.md](client-libraries)): A list of existing client libraries, both supported and user-contributed. ## Writing plugins -* **Authentication Plugins** ([docs/admin/authentication.md](../admin/authentication.html)): +* **Authentication Plugins** ([docs/admin/authentication.md](../admin/authentication)): The current and planned states of authentication tokens. -* **Authorization Plugins** ([docs/admin/authorization.md](../admin/authorization.html)): +* **Authorization Plugins** ([docs/admin/authorization.md](../admin/authorization)): Authorization applies to all HTTP requests on the main apiserver port. This doc explains the available authorization implementations. -* **Admission Control Plugins** ([admission_control](../design/admission_control.html)) +* **Admission Control Plugins** ([admission_control](../design/admission_control)) ## Building releases -* **Making release notes** ([making-release-notes.md](making-release-notes.html)): Generating release nodes for a new release. +* **Making release notes** ([making-release-notes.md](making-release-notes)): Generating release nodes for a new release. -* **Releasing Kubernetes** ([releasing.md](releasing.html)): How to create a Kubernetes release (as in version) +* **Releasing Kubernetes** ([releasing.md](releasing)): How to create a Kubernetes release (as in version) and how the version information gets embedded into the built binaries. diff --git a/v1.1/docs/devel/api-conventions.md b/v1.1/docs/devel/api-conventions.md index f1abcf4bd2..d023b37fb3 100644 --- a/v1.1/docs/devel/api-conventions.md +++ b/v1.1/docs/devel/api-conventions.md @@ -1,719 +1,663 @@ ---- -title: "API Conventions" ---- - -API Conventions -=============== - -Updated: 9/20/2015 - -*This document is oriented at users who want a deeper understanding of the Kubernetes -API structure, and developers wanting to extend the Kubernetes API. An introduction to -using resources with kubectl can be found in [Working with resources](../user-guide/working-with-resources.html).* - -**Table of Contents** - - - - [Types (Kinds)](#types-kinds) - - [Resources](#resources) - - [Objects](#objects) - - [Metadata](#metadata) - - [Spec and Status](#spec-and-status) - - [Typical status properties](#typical-status-properties) - - [References to related objects](#references-to-related-objects) - - [Lists of named subobjects preferred over maps](#lists-of-named-subobjects-preferred-over-maps) - - [Constants](#constants) - - [Lists and Simple kinds](#lists-and-simple-kinds) - - [Differing Representations](#differing-representations) - - [Verbs on Resources](#verbs-on-resources) - - [PATCH operations](#patch-operations) - - [Strategic Merge Patch](#strategic-merge-patch) - - [List Operations](#list-operations) - - [Map Operations](#map-operations) - - [Idempotency](#idempotency) - - [Defaulting](#defaulting) - - [Late Initialization](#late-initialization) - - [Concurrency Control and Consistency](#concurrency-control-and-consistency) - - [Serialization Format](#serialization-format) - - [Units](#units) - - [Selecting Fields](#selecting-fields) - - [Object references](#object-references) - - [HTTP Status codes](#http-status-codes) - - [Success codes](#success-codes) - - [Error codes](#error-codes) - - [Response Status Kind](#response-status-kind) - - [Events](#events) - - [Naming conventions](#naming-conventions) - - [Label, selector, and annotation conventions](#label-selector-and-annotation-conventions) - - - -The conventions of the [Kubernetes API](../api.html) (and related APIs in the ecosystem) are intended to ease client development and ensure that configuration mechanisms can be implemented that work across a diverse set of use cases consistently. - -The general style of the Kubernetes API is RESTful - clients create, update, delete, or retrieve a description of an object via the standard HTTP verbs (POST, PUT, DELETE, and GET) - and those APIs preferentially accept and return JSON. Kubernetes also exposes additional endpoints for non-standard verbs and allows alternative content types. All of the JSON accepted and returned by the server has a schema, identified by the "kind" and "apiVersion" fields. Where relevant HTTP header fields exist, they should mirror the content of JSON fields, but the information should not be represented only in the HTTP header. - -The following terms are defined: - -* **Kind** the name of a particular object schema (e.g. the "Cat" and "Dog" kinds would have different attributes and properties) -* **Resource** a representation of a system entity, sent or retrieved as JSON via HTTP to the server. Resources are exposed via: - * Collections - a list of resources of the same type, which may be queryable - * Elements - an individual resource, addressable via a URL - -Each resource typically accepts and returns data of a single kind. A kind may be accepted or returned by multiple resources that reflect specific use cases. For instance, the kind "Pod" is exposed as a "pods" resource that allows end users to create, update, and delete pods, while a separate "pod status" resource (that acts on "Pod" kind) allows automated processes to update a subset of the fields in that resource. - -Resource collections should be all lowercase and plural, whereas kinds are CamelCase and singular. - - -## Types (Kinds) - -Kinds are grouped into three categories: - -1. **Objects** represent a persistent entity in the system. - - Creating an API object is a record of intent - once created, the system will work to ensure that resource exists. All API objects have common metadata. - - An object may have multiple resources that clients can use to perform specific actions that create, update, delete, or get. - - Examples: `Pod`, `ReplicationController`, `Service`, `Namespace`, `Node`. - -2. **Lists** are collections of **resources** of one (usually) or more (occasionally) kinds. - - Lists have a limited set of common metadata. All lists use the "items" field to contain the array of objects they return. - - Most objects defined in the system should have an endpoint that returns the full set of resources, as well as zero or more endpoints that return subsets of the full list. Some objects may be singletons (the current user, the system defaults) and may not have lists. - - In addition, all lists that return objects with labels should support label filtering (see [docs/user-guide/labels.md](../user-guide/labels.html), and most lists should support filtering by fields. - - Examples: PodLists, ServiceLists, NodeLists - - TODO: Describe field filtering below or in a separate doc. - -3. **Simple** kinds are used for specific actions on objects and for non-persistent entities. - - Given their limited scope, they have the same set of limited common metadata as lists. - - For instance, the "Status" kind is returned when errors occur and is not persisted in the system. - - Many simple resources are "subresources", which are rooted at API paths of specific resources. When resources wish to expose alternative actions or views that are closely coupled to a single resource, they should do so using new sub-resources. Common subresources include: - - * `/binding`: Used to bind a resource representing a user request (e.g., Pod, PersistentVolumeClaim) to a cluster infrastructure resource (e.g., Node, PersistentVolume). - * `/status`: Used to write just the status portion of a resource. For example, the `/pods` endpoint only allows updates to `metadata` and `spec`, since those reflect end-user intent. An automated process should be able to modify status for users to see by sending an updated Pod kind to the server to the "/pods/<name>/status" endpoint - the alternate endpoint allows different rules to be applied to the update, and access to be appropriately restricted. - * `/scale`: Used to read and write the count of a resource in a manner that is independent of the specific resource schema. - - Two additional subresources, `proxy` and `portforward`, provide access to cluster resources as described in [docs/user-guide/accessing-the-cluster.md](../user-guide/accessing-the-cluster.html). - -The standard REST verbs (defined below) MUST return singular JSON objects. Some API endpoints may deviate from the strict REST pattern and return resources that are not singular JSON objects, such as streams of JSON objects or unstructured text log data. - -The term "kind" is reserved for these "top-level" API types. The term "type" should be used for distinguishing sub-categories within objects or subobjects. - -### Resources - -All JSON objects returned by an API MUST have the following fields: - -* kind: a string that identifies the schema this object should have -* apiVersion: a string that identifies the version of the schema the object should have - -These fields are required for proper decoding of the object. They may be populated by the server by default from the specified URL path, but the client likely needs to know the values in order to construct the URL path. - -### Objects - -#### Metadata - -Every object kind MUST have the following metadata in a nested object field called "metadata": - -* namespace: a namespace is a DNS compatible subdomain that objects are subdivided into. The default namespace is 'default'. See [docs/user-guide/namespaces.md](../user-guide/namespaces.html) for more. -* name: a string that uniquely identifies this object within the current namespace (see [docs/user-guide/identifiers.md](../user-guide/identifiers.html)). This value is used in the path when retrieving an individual object. -* uid: a unique in time and space value (typically an RFC 4122 generated identifier, see [docs/user-guide/identifiers.md](../user-guide/identifiers.html)) used to distinguish between objects with the same name that have been deleted and recreated - -Every object SHOULD have the following metadata in a nested object field called "metadata": - -* resourceVersion: a string that identifies the internal version of this object that can be used by clients to determine when objects have changed. This value MUST be treated as opaque by clients and passed unmodified back to the server. Clients should not assume that the resource version has meaning across namespaces, different kinds of resources, or different servers. (see [concurrency control](#concurrency-control-and-consistency), below, for more details) -* generation: a sequence number representing a specific generation of the desired state. Set by the system and monotonically increasing, per-resource. May be compared, such as for RAW and WAW consistency. -* creationTimestamp: a string representing an RFC 3339 date of the date and time an object was created -* deletionTimestamp: a string representing an RFC 3339 date of the date and time after which this resource will be deleted. This field is set by the server when a graceful deletion is requested by the user, and is not directly settable by a client. The resource will be deleted (no longer visible from resource lists, and not reachable by name) after the time in this field. Once set, this value may not be unset or be set further into the future, although it may be shortened or the resource may be deleted prior to this time. -* labels: a map of string keys and values that can be used to organize and categorize objects (see [docs/user-guide/labels.md](../user-guide/labels.html)) -* annotations: a map of string keys and values that can be used by external tooling to store and retrieve arbitrary metadata about this object (see [docs/user-guide/annotations.md](../user-guide/annotations.html)) - -Labels are intended for organizational purposes by end users (select the pods that match this label query). Annotations enable third-party automation and tooling to decorate objects with additional metadata for their own use. - -#### Spec and Status - -By convention, the Kubernetes API makes a distinction between the specification of the desired state of an object (a nested object field called "spec") and the status of the object at the current time (a nested object field called "status"). The specification is a complete description of the desired state, including configuration settings provided by the user, [default values](#defaulting) expanded by the system, and properties initialized or otherwise changed after creation by other ecosystem components (e.g., schedulers, auto-scalers), and is persisted in stable storage with the API object. If the specification is deleted, the object will be purged from the system. The status summarizes the current state of the object in the system, and is usually persisted with the object by an automated processes but may be generated on the fly. At some cost and perhaps some temporary degradation in behavior, the status could be reconstructed by observation if it were lost. - -When a new version of an object is POSTed or PUT, the "spec" is updated and available immediately. Over time the system will work to bring the "status" into line with the "spec". The system will drive toward the most recent "spec" regardless of previous versions of that stanza. In other words, if a value is changed from 2 to 5 in one PUT and then back down to 3 in another PUT the system is not required to 'touch base' at 5 before changing the "status" to 3. In other words, the system's behavior is *level-based* rather than *edge-based*. This enables robust behavior in the presence of missed intermediate state changes. - -The Kubernetes API also serves as the foundation for the declarative configuration schema for the system. In order to facilitate level-based operation and expression of declarative configuration, fields in the specification should have declarative rather than imperative names and semantics -- they represent the desired state, not actions intended to yield the desired state. - -The PUT and POST verbs on objects will ignore the "status" values. A `/status` subresource is provided to enable system components to update statuses of resources they manage. - -Otherwise, PUT expects the whole object to be specified. Therefore, if a field is omitted it is assumed that the client wants to clear that field's value. The PUT verb does not accept partial updates. Modification of just part of an object may be achieved by GETting the resource, modifying part of the spec, labels, or annotations, and then PUTting it back. See [concurrency control](#concurrency-control-and-consistency), below, regarding read-modify-write consistency when using this pattern. Some objects may expose alternative resource representations that allow mutation of the status, or performing custom actions on the object. - -All objects that represent a physical resource whose state may vary from the user's desired intent SHOULD have a "spec" and a "status". Objects whose state cannot vary from the user's desired intent MAY have only "spec", and MAY rename "spec" to a more appropriate name. - -Objects that contain both spec and status should not contain additional top-level fields other than the standard metadata fields. - -##### Typical status properties - -**Conditions** represent the latest available observations of an object's current state. Objects may report multiple conditions, and new types of conditions may be added in the future. Therefore, conditions are represented using a list/slice, where all have similar structure. - -The `FooCondition` type for some resource type `Foo` may include a subset of the following fields, but must contain at least `type` and `status` fields: - -{% highlight go %} -{% raw %} - Type FooConditionType `json:"type" description:"type of Foo condition"` - Status ConditionStatus `json:"status" description:"status of the condition, one of True, False, Unknown"` - LastHeartbeatTime unversioned.Time `json:"lastHeartbeatTime,omitempty" description:"last time we got an update on a given condition"` - LastTransitionTime unversioned.Time `json:"lastTransitionTime,omitempty" description:"last time the condition transit from one status to another"` - Reason string `json:"reason,omitempty" description:"one-word CamelCase reason for the condition's last transition"` - Message string `json:"message,omitempty" description:"human-readable message indicating details about last transition"` -{% endraw %} -{% endhighlight %} - -Additional fields may be added in the future. - -Conditions should be added to explicitly convey properties that users and components care about rather than requiring those properties to be inferred from other observations. - -Condition status values may be `True`, `False`, or `Unknown`. The absence of a condition should be interpreted the same as `Unknown`. - -In general, condition values may change back and forth, but some condition transitions may be monotonic, depending on the resource and condition type. However, conditions are observations and not, themselves, state machines, nor do we define comprehensive state machines for objects, nor behaviors associated with state transitions. The system is level-based rather than edge-triggered, and should assume an Open World. - -A typical oscillating condition type is `Ready`, which indicates the object was believed to be fully operational at the time it was last probed. A possible monotonic condition could be `Succeeded`. A `False` status for `Succeeded` would imply failure. An object that was still active would not have a `Succeeded` condition, or its status would be `Unknown`. - -Some resources in the v1 API contain fields called **`phase`**, and associated `message`, `reason`, and other status fields. The pattern of using `phase` is deprecated. Newer API types should use conditions instead. Phase was essentially a state-machine enumeration field, that contradicted [system-design principles](../design/principles.html#control-logic) and hampered evolution, since [adding new enum values breaks backward compatibility](api_changes.html). Rather than encouraging clients to infer implicit properties from phases, we intend to explicitly expose the conditions that clients need to monitor. Conditions also have the benefit that it is possible to create some conditions with uniform meaning across all resource types, while still exposing others that are unique to specific resource types. See [#7856](http://issues.k8s.io/7856) for more details and discussion. - -In condition types, and everywhere else they appear in the API, **`Reason`** is intended to be a one-word, CamelCase representation of the category of cause of the current status, and **`Message`** is intended to be a human-readable phrase or sentence, which may contain specific details of the individual occurrence. `Reason` is intended to be used in concise output, such as one-line `kubectl get` output, and in summarizing occurrences of causes, whereas `Message` is intended to be presented to users in detailed status explanations, such as `kubectl describe` output. - -Historical information status (e.g., last transition time, failure counts) is only provided with reasonable effort, and is not guaranteed to not be lost. - -Status information that may be large (especially proportional in size to collections of other resources, such as lists of references to other objects -- see below) and/or rapidly changing, such as [resource usage](../design/resources.html#usage-data), should be put into separate objects, with possibly a reference from the original object. This helps to ensure that GETs and watch remain reasonably efficient for the majority of clients, which may not need that data. - -Some resources report the `observedGeneration`, which is the `generation` most recently observed by the component responsible for acting upon changes to the desired state of the resource. This can be used, for instance, to ensure that the reported status reflects the most recent desired status. - -#### References to related objects - -References to loosely coupled sets of objects, such as [pods](../user-guide/pods.html) overseen by a [replication controller](../user-guide/replication-controller.html), are usually best referred to using a [label selector](../user-guide/labels.html). In order to ensure that GETs of individual objects remain bounded in time and space, these sets may be queried via separate API queries, but will not be expanded in the referring object's status. - -References to specific objects, especially specific resource versions and/or specific fields of those objects, are specified using the `ObjectReference` type (or other types representing strict subsets of it). Unlike partial URLs, the ObjectReference type facilitates flexible defaulting of fields from the referring object or other contextual information. - -References in the status of the referee to the referrer may be permitted, when the references are one-to-one and do not need to be frequently updated, particularly in an edge-based manner. - -#### Lists of named subobjects preferred over maps - -Discussed in [#2004](http://issue.k8s.io/2004) and elsewhere. There are no maps of subobjects in any API objects. Instead, the convention is to use a list of subobjects containing name fields. - -For example: - -{% highlight yaml %} -{% raw %} -ports: - - name: www - containerPort: 80 -{% endraw %} -{% endhighlight %} - -vs. - -{% highlight yaml %} -{% raw %} -ports: - www: - containerPort: 80 -{% endraw %} -{% endhighlight %} - -This rule maintains the invariant that all JSON/YAML keys are fields in API objects. The only exceptions are pure maps in the API (currently, labels, selectors, annotations, data), as opposed to sets of subobjects. - -#### Constants - -Some fields will have a list of allowed values (enumerations). These values will be strings, and they will be in CamelCase, with an initial uppercase letter. Examples: "ClusterFirst", "Pending", "ClientIP". - -### Lists and Simple kinds - -Every list or simple kind SHOULD have the following metadata in a nested object field called "metadata": - -* resourceVersion: a string that identifies the common version of the objects returned by in a list. This value MUST be treated as opaque by clients and passed unmodified back to the server. A resource version is only valid within a single namespace on a single kind of resource. - -Every simple kind returned by the server, and any simple kind sent to the server that must support idempotency or optimistic concurrency should return this value.Since simple resources are often used as input alternate actions that modify objects, the resource version of the simple resource should correspond to the resource version of the object. - - -## Differing Representations - -An API may represent a single entity in different ways for different clients, or transform an object after certain transitions in the system occur. In these cases, one request object may have two representations available as different resources, or different kinds. - -An example is a Service, which represents the intent of the user to group a set of pods with common behavior on common ports. When Kubernetes detects a pod matches the service selector, the IP address and port of the pod are added to an Endpoints resource for that Service. The Endpoints resource exists only if the Service exists, but exposes only the IPs and ports of the selected pods. The full service is represented by two distinct resources - under the original Service resource the user created, as well as in the Endpoints resource. - -As another example, a "pod status" resource may accept a PUT with the "pod" kind, with different rules about what fields may be changed. - -Future versions of Kubernetes may allow alternative encodings of objects beyond JSON. - - -## Verbs on Resources - -API resources should use the traditional REST pattern: - -* GET /<resourceNamePlural> - Retrieve a list of type <resourceName>, e.g. GET /pods returns a list of Pods. -* POST /<resourceNamePlural> - Create a new resource from the JSON object provided by the client. -* GET /<resourceNamePlural>/<name> - Retrieves a single resource with the given name, e.g. GET /pods/first returns a Pod named 'first'. Should be constant time, and the resource should be bounded in size. -* DELETE /<resourceNamePlural>/<name> - Delete the single resource with the given name. DeleteOptions may specify gracePeriodSeconds, the optional duration in seconds before the object should be deleted. Individual kinds may declare fields which provide a default grace period, and different kinds may have differing kind-wide default grace periods. A user provided grace period overrides a default grace period, including the zero grace period ("now"). -* PUT /<resourceNamePlural>/<name> - Update or create the resource with the given name with the JSON object provided by the client. -* PATCH /<resourceNamePlural>/<name> - Selectively modify the specified fields of the resource. See more information [below](#patch). -* GET /<resourceNamePlural>&watch=true - Receive a stream of JSON objects corresponding to changes made to any resource of the given kind over time. - -### PATCH operations - -The API supports three different PATCH operations, determined by their corresponding Content-Type header: - -* JSON Patch, `Content-Type: application/json-patch+json` - * As defined in [RFC6902](https://tools.ietf.org/html/rfc6902), a JSON Patch is a sequence of operations that are executed on the resource, e.g. `{"op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ]}`. For more details on how to use JSON Patch, see the RFC. -* Merge Patch, `Content-Type: application/merge-patch+json` - * As defined in [RFC7386](https://tools.ietf.org/html/rfc7386), a Merge Patch is essentially a partial representation of the resource. The submitted JSON is "merged" with the current resource to create a new one, then the new one is saved. For more details on how to use Merge Patch, see the RFC. -* Strategic Merge Patch, `Content-Type: application/strategic-merge-patch+json` - * Strategic Merge Patch is a custom implementation of Merge Patch. For a detailed explanation of how it works and why it needed to be introduced, see below. - -#### Strategic Merge Patch - -In the standard JSON merge patch, JSON objects are always merged but lists are always replaced. Often that isn't what we want. Let's say we start with the following Pod: - -{% highlight yaml %} -{% raw %} -spec: - containers: - - name: nginx - image: nginx-1.0 -{% endraw %} -{% endhighlight %} - -...and we POST that to the server (as JSON). Then let's say we want to *add* a container to this Pod. - -{% highlight yaml %} -{% raw %} -PATCH /api/v1/namespaces/default/pods/pod-name -spec: - containers: - - name: log-tailer - image: log-tailer-1.0 -{% endraw %} -{% endhighlight %} - -If we were to use standard Merge Patch, the entire container list would be replaced with the single log-tailer container. However, our intent is for the container lists to merge together based on the `name` field. - -To solve this problem, Strategic Merge Patch uses metadata attached to the API objects to determine what lists should be merged and which ones should not. Currently the metadata is available as struct tags on the API objects themselves, but will become available to clients as Swagger annotations in the future. In the above example, the `patchStrategy` metadata for the `containers` field would be `merge` and the `patchMergeKey` would be `name`. - -Note: If the patch results in merging two lists of scalars, the scalars are first deduplicated and then merged. - -Strategic Merge Patch also supports special operations as listed below. - -### List Operations - -To override the container list to be strictly replaced, regardless of the default: - -{% highlight yaml %} -{% raw %} -containers: - - name: nginx - image: nginx-1.0 - - $patch: replace # any further $patch operations nested in this list will be ignored -{% endraw %} -{% endhighlight %} - -To delete an element of a list that should be merged: - -{% highlight yaml %} -{% raw %} -containers: - - name: nginx - image: nginx-1.0 - - $patch: delete - name: log-tailer # merge key and value goes here -{% endraw %} -{% endhighlight %} - -### Map Operations - -To indicate that a map should not be merged and instead should be taken literally: - -{% highlight yaml %} -{% raw %} -$patch: replace # recursive and applies to all fields of the map it's in -containers: -- name: nginx - image: nginx-1.0 -{% endraw %} -{% endhighlight %} - -To delete a field of a map: - -{% highlight yaml %} -{% raw %} -name: nginx -image: nginx-1.0 -labels: - live: null # set the value of the map key to null -{% endraw %} -{% endhighlight %} - - -## Idempotency - -All compatible Kubernetes APIs MUST support "name idempotency" and respond with an HTTP status code 409 when a request is made to POST an object that has the same name as an existing object in the system. See [docs/user-guide/identifiers.md](../user-guide/identifiers.html) for details. - -Names generated by the system may be requested using `metadata.generateName`. GenerateName indicates that the name should be made unique by the server prior to persisting it. A non-empty value for the field indicates the name will be made unique (and the name returned to the client will be different than the name passed). The value of this field will be combined with a unique suffix on the server if the Name field has not been provided. The provided value must be valid within the rules for Name, and may be truncated by the length of the suffix required to make the value unique on the server. If this field is specified, and Name is not present, the server will NOT return a 409 if the generated name exists - instead, it will either return 201 Created or 504 with Reason `ServerTimeout` indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header). - -## Defaulting - -Default resource values are API version-specific, and they are applied during -the conversion from API-versioned declarative configuration to internal objects -representing the desired state (`Spec`) of the resource. Subsequent GETs of the -resource will include the default values explicitly. - -Incorporating the default values into the `Spec` ensures that `Spec` depicts the -full desired state so that it is easier for the system to determine how to -achieve the state, and for the user to know what to anticipate. - -API version-specific default values are set by the API server. - -## Late Initialization - -Late initialization is when resource fields are set by a system controller -after an object is created/updated. - -For example, the scheduler sets the `pod.spec.nodeName` field after the pod is created. - -Late-initializers should only make the following types of modifications: - - Setting previously unset fields - - Adding keys to maps - - Adding values to arrays which have mergeable semantics (`patchStrategy:"merge"` attribute in - the type definition). - -These conventions: - 1. allow a user (with sufficient privilege) to override any system-default behaviors by setting - the fields that would otherwise have been defaulted. - 1. enables updates from users to be merged with changes made during late initialization, using - strategic merge patch, as opposed to clobbering the change. - 1. allow the component which does the late-initialization to use strategic merge patch, which - facilitates composition and concurrency of such components. - -Although the apiserver Admission Control stage acts prior to object creation, -Admission Control plugins should follow the Late Initialization conventions -too, to allow their implementation to be later moved to a 'controller', or to client libraries. - -## Concurrency Control and Consistency - -Kubernetes leverages the concept of *resource versions* to achieve optimistic concurrency. All Kubernetes resources have a "resourceVersion" field as part of their metadata. This resourceVersion is a string that identifies the internal version of an object that can be used by clients to determine when objects have changed. When a record is about to be updated, it's version is checked against a pre-saved value, and if it doesn't match, the update fails with a StatusConflict (HTTP status code 409). - -The resourceVersion is changed by the server every time an object is modified. If resourceVersion is included with the PUT operation the system will verify that there have not been other successful mutations to the resource during a read/modify/write cycle, by verifying that the current value of resourceVersion matches the specified value. - -The resourceVersion is currently backed by [etcd's modifiedIndex](https://coreos.com/docs/distributed-configuration/etcd-api/). However, it's important to note that the application should *not* rely on the implementation details of the versioning system maintained by Kubernetes. We may change the implementation of resourceVersion in the future, such as to change it to a timestamp or per-object counter. - -The only way for a client to know the expected value of resourceVersion is to have received it from the server in response to a prior operation, typically a GET. This value MUST be treated as opaque by clients and passed unmodified back to the server. Clients should not assume that the resource version has meaning across namespaces, different kinds of resources, or different servers. Currently, the value of resourceVersion is set to match etcd's sequencer. You could think of it as a logical clock the API server can use to order requests. However, we expect the implementation of resourceVersion to change in the future, such as in the case we shard the state by kind and/or namespace, or port to another storage system. - -In the case of a conflict, the correct client action at this point is to GET the resource again, apply the changes afresh, and try submitting again. This mechanism can be used to prevent races like the following: - -``` -{% raw %} -Client #1 Client #2 -GET Foo GET Foo -Set Foo.Bar = "one" Set Foo.Baz = "two" -PUT Foo PUT Foo -{% endraw %} -``` - -When these sequences occur in parallel, either the change to Foo.Bar or the change to Foo.Baz can be lost. - -On the other hand, when specifying the resourceVersion, one of the PUTs will fail, since whichever write succeeds changes the resourceVersion for Foo. - -resourceVersion may be used as a precondition for other operations (e.g., GET, DELETE) in the future, such as for read-after-write consistency in the presence of caching. - -"Watch" operations specify resourceVersion using a query parameter. It is used to specify the point at which to begin watching the specified resources. This may be used to ensure that no mutations are missed between a GET of a resource (or list of resources) and a subsequent Watch, even if the current version of the resource is more recent. This is currently the main reason that list operations (GET on a collection) return resourceVersion. - - -## Serialization Format - -APIs may return alternative representations of any resource in response to an Accept header or under alternative endpoints, but the default serialization for input and output of API responses MUST be JSON. - -All dates should be serialized as RFC3339 strings. - -## Units - -Units must either be explicit in the field name (e.g., `timeoutSeconds`), or must be specified as part of the value (e.g., `resource.Quantity`). Which approach is preferred is TBD, though currently we use the `fooSeconds` convention for durations. - - -## Selecting Fields - -Some APIs may need to identify which field in a JSON object is invalid, or to reference a value to extract from a separate resource. The current recommendation is to use standard JavaScript syntax for accessing that field, assuming the JSON object was transformed into a JavaScript object, without the leading dot, such as `metadata.name`. - -Examples: - -* Find the field "current" in the object "state" in the second item in the array "fields": `fields[1].state.current` - -## Object references - -Object references should either be called `fooName` if referring to an object of kind `Foo` by just the name (within the current namespace, if a namespaced resource), or should be called `fooRef`, and should contain a subset of the fields of the `ObjectReference` type. - - -TODO: Plugins, extensions, nested kinds, headers - - -## HTTP Status codes - -The server will respond with HTTP status codes that match the HTTP spec. See the section below for a breakdown of the types of status codes the server will send. - -The following HTTP status codes may be returned by the API. - -#### Success codes - -* `200 StatusOK` - * Indicates that the request completed successfully. -* `201 StatusCreated` - * Indicates that the request to create kind completed successfully. -* `204 StatusNoContent` - * Indicates that the request completed successfully, and the response contains no body. - * Returned in response to HTTP OPTIONS requests. - -#### Error codes - -* `307 StatusTemporaryRedirect` - * Indicates that the address for the requested resource has changed. - * Suggested client recovery behavior - * Follow the redirect. -* `400 StatusBadRequest` - * Indicates the requested is invalid. - * Suggested client recovery behavior: - * Do not retry. Fix the request. -* `401 StatusUnauthorized` - * Indicates that the server can be reached and understood the request, but refuses to take any further action, because the client must provide authorization. If the client has provided authorization, the server is indicating the provided authorization is unsuitable or invalid. - * Suggested client recovery behavior - * If the user has not supplied authorization information, prompt them for the appropriate credentials - * If the user has supplied authorization information, inform them their credentials were rejected and optionally prompt them again. -* `403 StatusForbidden` - * Indicates that the server can be reached and understood the request, but refuses to take any further action, because it is configured to deny access for some reason to the requested resource by the client. - * Suggested client recovery behavior - * Do not retry. Fix the request. -* `404 StatusNotFound` - * Indicates that the requested resource does not exist. - * Suggested client recovery behavior - * Do not retry. Fix the request. -* `405 StatusMethodNotAllowed` - * Indicates that the action the client attempted to perform on the resource was not supported by the code. - * Suggested client recovery behavior - * Do not retry. Fix the request. -* `409 StatusConflict` - * Indicates that either the resource the client attempted to create already exists or the requested update operation cannot be completed due to a conflict. - * Suggested client recovery behavior - * * If creating a new resource - * * Either change the identifier and try again, or GET and compare the fields in the pre-existing object and issue a PUT/update to modify the existing object. - * * If updating an existing resource: - * See `Conflict` from the `status` response section below on how to retrieve more information about the nature of the conflict. - * GET and compare the fields in the pre-existing object, merge changes (if still valid according to preconditions), and retry with the updated request (including `ResourceVersion`). -* `422 StatusUnprocessableEntity` - * Indicates that the requested create or update operation cannot be completed due to invalid data provided as part of the request. - * Suggested client recovery behavior - * Do not retry. Fix the request. -* `429 StatusTooManyRequests` - * Indicates that the either the client rate limit has been exceeded or the server has received more requests then it can process. - * Suggested client recovery behavior: - * Read the `Retry-After` HTTP header from the response, and wait at least that long before retrying. -* `500 StatusInternalServerError` - * Indicates that the server can be reached and understood the request, but either an unexpected internal error occurred and the outcome of the call is unknown, or the server cannot complete the action in a reasonable time (this maybe due to temporary server load or a transient communication issue with another server). - * Suggested client recovery behavior: - * Retry with exponential backoff. -* `503 StatusServiceUnavailable` - * Indicates that required service is unavailable. - * Suggested client recovery behavior: - * Retry with exponential backoff. -* `504 StatusServerTimeout` - * Indicates that the request could not be completed within the given time. Clients can get this response ONLY when they specified a timeout param in the request. - * Suggested client recovery behavior: - * Increase the value of the timeout param and retry with exponential backoff - -## Response Status Kind - -Kubernetes will always return the `Status` kind from any API endpoint when an error occurs. -Clients SHOULD handle these types of objects when appropriate. - -A `Status` kind will be returned by the API in two cases: - * When an operation is not successful (i.e. when the server would return a non 2xx HTTP status code). - * When a HTTP `DELETE` call is successful. - -The status object is encoded as JSON and provided as the body of the response. The status object contains fields for humans and machine consumers of the API to get more detailed information for the cause of the failure. The information in the status object supplements, but does not override, the HTTP status code's meaning. When fields in the status object have the same meaning as generally defined HTTP headers and that header is returned with the response, the header should be considered as having higher priority. - -**Example:** - -{% highlight console %} -{% raw %} -$ curl -v -k -H "Authorization: Bearer WhCDvq4VPpYhrcfmF6ei7V9qlbqTubUc" https://10.240.122.184:443/api/v1/namespaces/default/pods/grafana - -> GET /api/v1/namespaces/default/pods/grafana HTTP/1.1 -> User-Agent: curl/7.26.0 -> Host: 10.240.122.184 -> Accept: */* -> Authorization: Bearer WhCDvq4VPpYhrcfmF6ei7V9qlbqTubUc -> - -< HTTP/1.1 404 Not Found -< Content-Type: application/json -< Date: Wed, 20 May 2015 18:10:42 GMT -< Content-Length: 232 -< -{ - "kind": "Status", - "apiVersion": "v1", - "metadata": {}, - "status": "Failure", - "message": "pods \"grafana\" not found", - "reason": "NotFound", - "details": { - "name": "grafana", - "kind": "pods" - }, - "code": 404 -} -{% endraw %} -{% endhighlight %} - -`status` field contains one of two possible values: -* `Success` -* `Failure` - -`message` may contain human-readable description of the error - -`reason` may contain a machine-readable, one-word, CamelCase description of why this operation is in the `Failure` status. If this value is empty there is no information available. The `reason` clarifies an HTTP status code but does not override it. - -`details` may contain extended data associated with the reason. Each reason may define its own extended details. This field is optional and the data returned is not guaranteed to conform to any schema except that defined by the reason type. - -Possible values for the `reason` and `details` fields: -* `BadRequest` - * Indicates that the request itself was invalid, because the request doesn't make any sense, for example deleting a read-only object. - * This is different than `status reason` `Invalid` above which indicates that the API call could possibly succeed, but the data was invalid. - * API calls that return BadRequest can never succeed. - * Http status code: `400 StatusBadRequest` -* `Unauthorized` - * Indicates that the server can be reached and understood the request, but refuses to take any further action without the client providing appropriate authorization. If the client has provided authorization, this error indicates the provided credentials are insufficient or invalid. - * Details (optional): - * `kind string` - * The kind attribute of the unauthorized resource (on some operations may differ from the requested resource). - * `name string` - * The identifier of the unauthorized resource. - * HTTP status code: `401 StatusUnauthorized` -* `Forbidden` - * Indicates that the server can be reached and understood the request, but refuses to take any further action, because it is configured to deny access for some reason to the requested resource by the client. - * Details (optional): - * `kind string` - * The kind attribute of the forbidden resource (on some operations may differ from the requested resource). - * `name string` - * The identifier of the forbidden resource. - * HTTP status code: `403 StatusForbidden` -* `NotFound` - * Indicates that one or more resources required for this operation could not be found. - * Details (optional): - * `kind string` - * The kind attribute of the missing resource (on some operations may differ from the requested resource). - * `name string` - * The identifier of the missing resource. - * HTTP status code: `404 StatusNotFound` -* `AlreadyExists` - * Indicates that the resource you are creating already exists. - * Details (optional): - * `kind string` - * The kind attribute of the conflicting resource. - * `name string` - * The identifier of the conflicting resource. - * HTTP status code: `409 StatusConflict` -* `Conflict` - * Indicates that the requested update operation cannot be completed due to a conflict. The client may need to alter the request. Each resource may define custom details that indicate the nature of the conflict. - * HTTP status code: `409 StatusConflict` -* `Invalid` - * Indicates that the requested create or update operation cannot be completed due to invalid data provided as part of the request. - * Details (optional): - * `kind string` - * the kind attribute of the invalid resource - * `name string` - * the identifier of the invalid resource - * `causes` - * One or more `StatusCause` entries indicating the data in the provided resource that was invalid. The `reason`, `message`, and `field` attributes will be set. - * HTTP status code: `422 StatusUnprocessableEntity` -* `Timeout` - * Indicates that the request could not be completed within the given time. Clients may receive this response if the server has decided to rate limit the client, or if the server is overloaded and cannot process the request at this time. - * Http status code: `429 TooManyRequests` - * The server should set the `Retry-After` HTTP header and return `retryAfterSeconds` in the details field of the object. A value of `0` is the default. -* `ServerTimeout` - * Indicates that the server can be reached and understood the request, but cannot complete the action in a reasonable time. This maybe due to temporary server load or a transient communication issue with another server. - * Details (optional): - * `kind string` - * The kind attribute of the resource being acted on. - * `name string` - * The operation that is being attempted. - * The server should set the `Retry-After` HTTP header and return `retryAfterSeconds` in the details field of the object. A value of `0` is the default. - * Http status code: `504 StatusServerTimeout` -* `MethodNotAllowed` - * Indicates that the action the client attempted to perform on the resource was not supported by the code. - * For instance, attempting to delete a resource that can only be created. - * API calls that return MethodNotAllowed can never succeed. - * Http status code: `405 StatusMethodNotAllowed` -* `InternalError` - * Indicates that an internal error occurred, it is unexpected and the outcome of the call is unknown. - * Details (optional): - * `causes` - * The original error. - * Http status code: `500 StatusInternalServerError` - -`code` may contain the suggested HTTP return code for this status. - - -## Events - -Events are complementary to status information, since they can provide some historical information about status and occurrences in addition to current or previous status. Generate events for situations users or administrators should be alerted about. - -Choose a unique, specific, short, CamelCase reason for each event category. For example, `FreeDiskSpaceInvalid` is a good event reason because it is likely to refer to just one situation, but `Started` is not a good reason because it doesn't sufficiently indicate what started, even when combined with other event fields. - -`Error creating foo` or `Error creating foo %s` would be appropriate for an event message, with the latter being preferable, since it is more informational. - -Accumulate repeated events in the client, especially for frequent events, to reduce data volume, load on the system, and noise exposed to users. - -## Naming conventions - -* Go field names must be CamelCase. JSON field names must be camelCase. Other than capitalization of the initial letter, the two should almost always match. No underscores nor dashes in either. -* Field and resource names should be declarative, not imperative (DoSomething, SomethingDoer, DoneBy, DoneAt). -* `Minion` has been deprecated in favor of `Node`. Use `Node` where referring to the node resource in the context of the cluster. Use `Host` where referring to properties of the individual physical/virtual system, such as `hostname`, `hostPath`, `hostNetwork`, etc. -* `FooController` is a deprecated kind naming convention. Name the kind after the thing being controlled instead (e.g., `Job` rather than `JobController`). -* The name of a field that specifies the time at which `something` occurs should be called `somethingTime`. Do not use `stamp` (e.g., `creationTimestamp`). -* We use the `fooSeconds` convention for durations, as discussed in the [units subsection](#units). - * `fooPeriodSeconds` is preferred for periodic intervals and other waiting periods (e.g., over `fooIntervalSeconds`). - * `fooTimeoutSeconds` is preferred for inactivity/unresponsiveness deadlines. - * `fooDeadlineSeconds` is preferred for activity completion deadlines. -* Do not use abbreviations in the API, except where they are extremely commonly used, such as "id", "args", or "stdin". -* Acronyms should similarly only be used when extremely commonly known. All letters in the acronym should have the same case, using the appropriate case for the situation. For example, at the beginning of a field name, the acronym should be all lowercase, such as "httpGet". Where used as a constant, all letters should be uppercase, such as "TCP" or "UDP". -* The name of a field referring to another resource of kind `Foo` by name should be called `fooName`. The name of a field referring to another resource of kind `Foo` by ObjectReference (or subset thereof) should be called `fooRef`. -* More generally, include the units and/or type in the field name if they could be ambiguous and they are not specified by the value or value type. - -## Label, selector, and annotation conventions - -Labels are the domain of users. They are intended to facilitate organization and management of API resources using attributes that are meaningful to users, as opposed to meaningful to the system. Think of them as user-created mp3 or email inbox labels, as opposed to the directory structure used by a program to store its data. The former is enables the user to apply an arbitrary ontology, whereas the latter is implementation-centric and inflexible. Users will use labels to select resources to operate on, display label values in CLI/UI columns, etc. Users should always retain full power and flexibility over the label schemas they apply to labels in their namespaces. - -However, we should support conveniences for common cases by default. For example, what we now do in ReplicationController is automatically set the RC's selector and labels to the labels in the pod template by default, if they are not already set. That ensures that the selector will match the template, and that the RC can be managed using the same labels as the pods it creates. Note that once we generalize selectors, it won't necessarily be possible to unambiguously generate labels that match an arbitrary selector. - -If the user wants to apply additional labels to the pods that it doesn't select upon, such as to facilitate adoption of pods or in the expectation that some label values will change, they can set the selector to a subset of the pod labels. Similarly, the RC's labels could be initialized to a subset of the pod template's labels, or could include additional/different labels. - -For disciplined users managing resources within their own namespaces, it's not that hard to consistently apply schemas that ensure uniqueness. One just needs to ensure that at least one value of some label key in common differs compared to all other comparable resources. We could/should provide a verification tool to check that. However, development of conventions similar to the examples in [Labels](../user-guide/labels.html) make uniqueness straightforward. Furthermore, relatively narrowly used namespaces (e.g., per environment, per application) can be used to reduce the set of resources that could potentially cause overlap. - -In cases where users could be running misc. examples with inconsistent schemas, or where tooling or components need to programmatically generate new objects to be selected, there needs to be a straightforward way to generate unique label sets. A simple way to ensure uniqueness of the set is to ensure uniqueness of a single label value, such as by using a resource name, uid, resource hash, or generation number. - -Problems with uids and hashes, however, include that they have no semantic meaning to the user, are not memorable nor readily recognizable, and are not predictable. Lack of predictability obstructs use cases such as creation of a replication controller from a pod, such as people want to do when exploring the system, bootstrapping a self-hosted cluster, or deletion and re-creation of a new RC that adopts the pods of the previous one, such as to rename it. Generation numbers are more predictable and much clearer, assuming there is a logical sequence. Fortunately, for deployments that's the case. For jobs, use of creation timestamps is common internally. Users should always be able to turn off auto-generation, in order to permit some of the scenarios described above. Note that auto-generated labels will also become one more field that needs to be stripped out when cloning a resource, within a namespace, in a new namespace, in a new cluster, etc., and will need to be ignored around when updating a resource via patch or read-modify-write sequence. - -Inclusion of a system prefix in a label key is fairly hostile to UX. A prefix is only necessary in the case that the user cannot choose the label key, in order to avoid collisions with user-defined labels. However, I firmly believe that the user should always be allowed to select the label keys to use on their resources, so it should always be possible to override default label keys. - -Therefore, resources supporting auto-generation of unique labels should have a `uniqueLabelKey` field, so that the user could specify the key if they wanted to, but if unspecified, it could be set by default, such as to the resource type, like job, deployment, or replicationController. The value would need to be at least spatially unique, and perhaps temporally unique in the case of job. - -Annotations have very different intended usage from labels. We expect them to be primarily generated and consumed by tooling and system extensions. I'm inclined to generalize annotations to permit them to directly store arbitrary json. Rigid names and name prefixes make sense, since they are analogous to API fields. - -In fact, in-development API fields, including those used to represent fields of newer alpha/beta API versions in the older stable storage version, may be represented as annotations with the form `something.alpha.kubernetes.io/name` or `something.beta.kubernetes.io/name` (depending on our confidence in it). For example `net.alpha.kubernetes.io/policy` might represent an experimental network policy field. - -Other advice regarding use of labels, annotations, and other generic map keys by Kubernetes components and tools: - - Key names should be all lowercase, with words separated by dashes, such as `desired-replicas` - - Prefix the key with `kubernetes.io/` or `foo.kubernetes.io/`, preferably the latter if the label/annotation is specific to `foo` - - For instance, prefer `service-account.kubernetes.io/name` over `kubernetes.io/service-account.name` - - Use annotations to store API extensions that the controller responsible for the resource doesn't need to know about, experimental fields that aren't intended to be generally used API fields, etc. Beware that annotations aren't automatically handled by the API conversion machinery. - - - - +--- +title: "API Conventions" +--- + +API Conventions +=============== + +Updated: 9/20/2015 + +*This document is oriented at users who want a deeper understanding of the Kubernetes +API structure, and developers wanting to extend the Kubernetes API. An introduction to +using resources with kubectl can be found in [Working with resources](../user-guide/working-with-resources).* + +{% include pagetoc.html %} + + +The conventions of the [Kubernetes API](../api) (and related APIs in the ecosystem) are intended to ease client development and ensure that configuration mechanisms can be implemented that work across a diverse set of use cases consistently. + +The general style of the Kubernetes API is RESTful - clients create, update, delete, or retrieve a description of an object via the standard HTTP verbs (POST, PUT, DELETE, and GET) - and those APIs preferentially accept and return JSON. Kubernetes also exposes additional endpoints for non-standard verbs and allows alternative content types. All of the JSON accepted and returned by the server has a schema, identified by the "kind" and "apiVersion" fields. Where relevant HTTP header fields exist, they should mirror the content of JSON fields, but the information should not be represented only in the HTTP header. + +The following terms are defined: + +* **Kind** the name of a particular object schema (e.g. the "Cat" and "Dog" kinds would have different attributes and properties) +* **Resource** a representation of a system entity, sent or retrieved as JSON via HTTP to the server. Resources are exposed via: + * Collections - a list of resources of the same type, which may be queryable + * Elements - an individual resource, addressable via a URL + +Each resource typically accepts and returns data of a single kind. A kind may be accepted or returned by multiple resources that reflect specific use cases. For instance, the kind "Pod" is exposed as a "pods" resource that allows end users to create, update, and delete pods, while a separate "pod status" resource (that acts on "Pod" kind) allows automated processes to update a subset of the fields in that resource. + +Resource collections should be all lowercase and plural, whereas kinds are CamelCase and singular. + + +## Types (Kinds) + +Kinds are grouped into three categories: + +1. **Objects** represent a persistent entity in the system. + + Creating an API object is a record of intent - once created, the system will work to ensure that resource exists. All API objects have common metadata. + + An object may have multiple resources that clients can use to perform specific actions that create, update, delete, or get. + + Examples: `Pod`, `ReplicationController`, `Service`, `Namespace`, `Node`. + +2. **Lists** are collections of **resources** of one (usually) or more (occasionally) kinds. + + Lists have a limited set of common metadata. All lists use the "items" field to contain the array of objects they return. + + Most objects defined in the system should have an endpoint that returns the full set of resources, as well as zero or more endpoints that return subsets of the full list. Some objects may be singletons (the current user, the system defaults) and may not have lists. + + In addition, all lists that return objects with labels should support label filtering (see [docs/user-guide/labels.md](../user-guide/labels), and most lists should support filtering by fields. + + Examples: PodLists, ServiceLists, NodeLists + + TODO: Describe field filtering below or in a separate doc. + +3. **Simple** kinds are used for specific actions on objects and for non-persistent entities. + + Given their limited scope, they have the same set of limited common metadata as lists. + + For instance, the "Status" kind is returned when errors occur and is not persisted in the system. + + Many simple resources are "subresources", which are rooted at API paths of specific resources. When resources wish to expose alternative actions or views that are closely coupled to a single resource, they should do so using new sub-resources. Common subresources include: + + * `/binding`: Used to bind a resource representing a user request (e.g., Pod, PersistentVolumeClaim) to a cluster infrastructure resource (e.g., Node, PersistentVolume). + * `/status`: Used to write just the status portion of a resource. For example, the `/pods` endpoint only allows updates to `metadata` and `spec`, since those reflect end-user intent. An automated process should be able to modify status for users to see by sending an updated Pod kind to the server to the "/pods/<name>/status" endpoint - the alternate endpoint allows different rules to be applied to the update, and access to be appropriately restricted. + * `/scale`: Used to read and write the count of a resource in a manner that is independent of the specific resource schema. + + Two additional subresources, `proxy` and `portforward`, provide access to cluster resources as described in [docs/user-guide/accessing-the-cluster.md](../user-guide/accessing-the-cluster). + +The standard REST verbs (defined below) MUST return singular JSON objects. Some API endpoints may deviate from the strict REST pattern and return resources that are not singular JSON objects, such as streams of JSON objects or unstructured text log data. + +The term "kind" is reserved for these "top-level" API types. The term "type" should be used for distinguishing sub-categories within objects or subobjects. + +### Resources + +All JSON objects returned by an API MUST have the following fields: + +* kind: a string that identifies the schema this object should have +* apiVersion: a string that identifies the version of the schema the object should have + +These fields are required for proper decoding of the object. They may be populated by the server by default from the specified URL path, but the client likely needs to know the values in order to construct the URL path. + +### Objects + +#### Metadata + +Every object kind MUST have the following metadata in a nested object field called "metadata": + +* namespace: a namespace is a DNS compatible subdomain that objects are subdivided into. The default namespace is 'default'. See [docs/user-guide/namespaces.md](../user-guide/namespaces) for more. +* name: a string that uniquely identifies this object within the current namespace (see [docs/user-guide/identifiers.md](../user-guide/identifiers)). This value is used in the path when retrieving an individual object. +* uid: a unique in time and space value (typically an RFC 4122 generated identifier, see [docs/user-guide/identifiers.md](../user-guide/identifiers)) used to distinguish between objects with the same name that have been deleted and recreated + +Every object SHOULD have the following metadata in a nested object field called "metadata": + +* resourceVersion: a string that identifies the internal version of this object that can be used by clients to determine when objects have changed. This value MUST be treated as opaque by clients and passed unmodified back to the server. Clients should not assume that the resource version has meaning across namespaces, different kinds of resources, or different servers. (see [concurrency control](#concurrency-control-and-consistency), below, for more details) +* generation: a sequence number representing a specific generation of the desired state. Set by the system and monotonically increasing, per-resource. May be compared, such as for RAW and WAW consistency. +* creationTimestamp: a string representing an RFC 3339 date of the date and time an object was created +* deletionTimestamp: a string representing an RFC 3339 date of the date and time after which this resource will be deleted. This field is set by the server when a graceful deletion is requested by the user, and is not directly settable by a client. The resource will be deleted (no longer visible from resource lists, and not reachable by name) after the time in this field. Once set, this value may not be unset or be set further into the future, although it may be shortened or the resource may be deleted prior to this time. +* labels: a map of string keys and values that can be used to organize and categorize objects (see [docs/user-guide/labels.md](../user-guide/labels)) +* annotations: a map of string keys and values that can be used by external tooling to store and retrieve arbitrary metadata about this object (see [docs/user-guide/annotations.md](../user-guide/annotations)) + +Labels are intended for organizational purposes by end users (select the pods that match this label query). Annotations enable third-party automation and tooling to decorate objects with additional metadata for their own use. + +#### Spec and Status + +By convention, the Kubernetes API makes a distinction between the specification of the desired state of an object (a nested object field called "spec") and the status of the object at the current time (a nested object field called "status"). The specification is a complete description of the desired state, including configuration settings provided by the user, [default values](#defaulting) expanded by the system, and properties initialized or otherwise changed after creation by other ecosystem components (e.g., schedulers, auto-scalers), and is persisted in stable storage with the API object. If the specification is deleted, the object will be purged from the system. The status summarizes the current state of the object in the system, and is usually persisted with the object by an automated processes but may be generated on the fly. At some cost and perhaps some temporary degradation in behavior, the status could be reconstructed by observation if it were lost. + +When a new version of an object is POSTed or PUT, the "spec" is updated and available immediately. Over time the system will work to bring the "status" into line with the "spec". The system will drive toward the most recent "spec" regardless of previous versions of that stanza. In other words, if a value is changed from 2 to 5 in one PUT and then back down to 3 in another PUT the system is not required to 'touch base' at 5 before changing the "status" to 3. In other words, the system's behavior is *level-based* rather than *edge-based*. This enables robust behavior in the presence of missed intermediate state changes. + +The Kubernetes API also serves as the foundation for the declarative configuration schema for the system. In order to facilitate level-based operation and expression of declarative configuration, fields in the specification should have declarative rather than imperative names and semantics -- they represent the desired state, not actions intended to yield the desired state. + +The PUT and POST verbs on objects will ignore the "status" values. A `/status` subresource is provided to enable system components to update statuses of resources they manage. + +Otherwise, PUT expects the whole object to be specified. Therefore, if a field is omitted it is assumed that the client wants to clear that field's value. The PUT verb does not accept partial updates. Modification of just part of an object may be achieved by GETting the resource, modifying part of the spec, labels, or annotations, and then PUTting it back. See [concurrency control](#concurrency-control-and-consistency), below, regarding read-modify-write consistency when using this pattern. Some objects may expose alternative resource representations that allow mutation of the status, or performing custom actions on the object. + +All objects that represent a physical resource whose state may vary from the user's desired intent SHOULD have a "spec" and a "status". Objects whose state cannot vary from the user's desired intent MAY have only "spec", and MAY rename "spec" to a more appropriate name. + +Objects that contain both spec and status should not contain additional top-level fields other than the standard metadata fields. + +##### Typical status properties + +**Conditions** represent the latest available observations of an object's current state. Objects may report multiple conditions, and new types of conditions may be added in the future. Therefore, conditions are represented using a list/slice, where all have similar structure. + +The `FooCondition` type for some resource type `Foo` may include a subset of the following fields, but must contain at least `type` and `status` fields: + +{% highlight go %} + Type FooConditionType `json:"type" description:"type of Foo condition"` + Status ConditionStatus `json:"status" description:"status of the condition, one of True, False, Unknown"` + LastHeartbeatTime unversioned.Time `json:"lastHeartbeatTime,omitempty" description:"last time we got an update on a given condition"` + LastTransitionTime unversioned.Time `json:"lastTransitionTime,omitempty" description:"last time the condition transit from one status to another"` + Reason string `json:"reason,omitempty" description:"one-word CamelCase reason for the condition's last transition"` + Message string `json:"message,omitempty" description:"human-readable message indicating details about last transition"` +{% endhighlight %} + +Additional fields may be added in the future. + +Conditions should be added to explicitly convey properties that users and components care about rather than requiring those properties to be inferred from other observations. + +Condition status values may be `True`, `False`, or `Unknown`. The absence of a condition should be interpreted the same as `Unknown`. + +In general, condition values may change back and forth, but some condition transitions may be monotonic, depending on the resource and condition type. However, conditions are observations and not, themselves, state machines, nor do we define comprehensive state machines for objects, nor behaviors associated with state transitions. The system is level-based rather than edge-triggered, and should assume an Open World. + +A typical oscillating condition type is `Ready`, which indicates the object was believed to be fully operational at the time it was last probed. A possible monotonic condition could be `Succeeded`. A `False` status for `Succeeded` would imply failure. An object that was still active would not have a `Succeeded` condition, or its status would be `Unknown`. + +Some resources in the v1 API contain fields called **`phase`**, and associated `message`, `reason`, and other status fields. The pattern of using `phase` is deprecated. Newer API types should use conditions instead. Phase was essentially a state-machine enumeration field, that contradicted [system-design principles](../design/principles.html#control-logic) and hampered evolution, since [adding new enum values breaks backward compatibility](api_changes). Rather than encouraging clients to infer implicit properties from phases, we intend to explicitly expose the conditions that clients need to monitor. Conditions also have the benefit that it is possible to create some conditions with uniform meaning across all resource types, while still exposing others that are unique to specific resource types. See [#7856](http://issues.k8s.io/7856) for more details and discussion. + +In condition types, and everywhere else they appear in the API, **`Reason`** is intended to be a one-word, CamelCase representation of the category of cause of the current status, and **`Message`** is intended to be a human-readable phrase or sentence, which may contain specific details of the individual occurrence. `Reason` is intended to be used in concise output, such as one-line `kubectl get` output, and in summarizing occurrences of causes, whereas `Message` is intended to be presented to users in detailed status explanations, such as `kubectl describe` output. + +Historical information status (e.g., last transition time, failure counts) is only provided with reasonable effort, and is not guaranteed to not be lost. + +Status information that may be large (especially proportional in size to collections of other resources, such as lists of references to other objects -- see below) and/or rapidly changing, such as [resource usage](../design/resources.html#usage-data), should be put into separate objects, with possibly a reference from the original object. This helps to ensure that GETs and watch remain reasonably efficient for the majority of clients, which may not need that data. + +Some resources report the `observedGeneration`, which is the `generation` most recently observed by the component responsible for acting upon changes to the desired state of the resource. This can be used, for instance, to ensure that the reported status reflects the most recent desired status. + +#### References to related objects + +References to loosely coupled sets of objects, such as [pods](../user-guide/pods) overseen by a [replication controller](../user-guide/replication-controller), are usually best referred to using a [label selector](../user-guide/labels). In order to ensure that GETs of individual objects remain bounded in time and space, these sets may be queried via separate API queries, but will not be expanded in the referring object's status. + +References to specific objects, especially specific resource versions and/or specific fields of those objects, are specified using the `ObjectReference` type (or other types representing strict subsets of it). Unlike partial URLs, the ObjectReference type facilitates flexible defaulting of fields from the referring object or other contextual information. + +References in the status of the referee to the referrer may be permitted, when the references are one-to-one and do not need to be frequently updated, particularly in an edge-based manner. + +#### Lists of named subobjects preferred over maps + +Discussed in [#2004](http://issue.k8s.io/2004) and elsewhere. There are no maps of subobjects in any API objects. Instead, the convention is to use a list of subobjects containing name fields. + +For example: + +{% highlight yaml %} +ports: + - name: www + containerPort: 80 +{% endhighlight %} + +vs. + +{% highlight yaml %} +ports: + www: + containerPort: 80 +{% endhighlight %} + +This rule maintains the invariant that all JSON/YAML keys are fields in API objects. The only exceptions are pure maps in the API (currently, labels, selectors, annotations, data), as opposed to sets of subobjects. + +#### Constants + +Some fields will have a list of allowed values (enumerations). These values will be strings, and they will be in CamelCase, with an initial uppercase letter. Examples: "ClusterFirst", "Pending", "ClientIP". + +### Lists and Simple kinds + +Every list or simple kind SHOULD have the following metadata in a nested object field called "metadata": + +* resourceVersion: a string that identifies the common version of the objects returned by in a list. This value MUST be treated as opaque by clients and passed unmodified back to the server. A resource version is only valid within a single namespace on a single kind of resource. + +Every simple kind returned by the server, and any simple kind sent to the server that must support idempotency or optimistic concurrency should return this value.Since simple resources are often used as input alternate actions that modify objects, the resource version of the simple resource should correspond to the resource version of the object. + + +## Differing Representations + +An API may represent a single entity in different ways for different clients, or transform an object after certain transitions in the system occur. In these cases, one request object may have two representations available as different resources, or different kinds. + +An example is a Service, which represents the intent of the user to group a set of pods with common behavior on common ports. When Kubernetes detects a pod matches the service selector, the IP address and port of the pod are added to an Endpoints resource for that Service. The Endpoints resource exists only if the Service exists, but exposes only the IPs and ports of the selected pods. The full service is represented by two distinct resources - under the original Service resource the user created, as well as in the Endpoints resource. + +As another example, a "pod status" resource may accept a PUT with the "pod" kind, with different rules about what fields may be changed. + +Future versions of Kubernetes may allow alternative encodings of objects beyond JSON. + + +## Verbs on Resources + +API resources should use the traditional REST pattern: + +* GET /<resourceNamePlural> - Retrieve a list of type <resourceName>, e.g. GET /pods returns a list of Pods. +* POST /<resourceNamePlural> - Create a new resource from the JSON object provided by the client. +* GET /<resourceNamePlural>/<name> - Retrieves a single resource with the given name, e.g. GET /pods/first returns a Pod named 'first'. Should be constant time, and the resource should be bounded in size. +* DELETE /<resourceNamePlural>/<name> - Delete the single resource with the given name. DeleteOptions may specify gracePeriodSeconds, the optional duration in seconds before the object should be deleted. Individual kinds may declare fields which provide a default grace period, and different kinds may have differing kind-wide default grace periods. A user provided grace period overrides a default grace period, including the zero grace period ("now"). +* PUT /<resourceNamePlural>/<name> - Update or create the resource with the given name with the JSON object provided by the client. +* PATCH /<resourceNamePlural>/<name> - Selectively modify the specified fields of the resource. See more information [below](#patch). +* GET /<resourceNamePlural>&watch=true - Receive a stream of JSON objects corresponding to changes made to any resource of the given kind over time. + +### PATCH operations + +The API supports three different PATCH operations, determined by their corresponding Content-Type header: + +* JSON Patch, `Content-Type: application/json-patch+json` + * As defined in [RFC6902](https://tools.ietf.org/html/rfc6902), a JSON Patch is a sequence of operations that are executed on the resource, e.g. `{"op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ]}`. For more details on how to use JSON Patch, see the RFC. +* Merge Patch, `Content-Type: application/merge-patch+json` + * As defined in [RFC7386](https://tools.ietf.org/html/rfc7386), a Merge Patch is essentially a partial representation of the resource. The submitted JSON is "merged" with the current resource to create a new one, then the new one is saved. For more details on how to use Merge Patch, see the RFC. +* Strategic Merge Patch, `Content-Type: application/strategic-merge-patch+json` + * Strategic Merge Patch is a custom implementation of Merge Patch. For a detailed explanation of how it works and why it needed to be introduced, see below. + +#### Strategic Merge Patch + +In the standard JSON merge patch, JSON objects are always merged but lists are always replaced. Often that isn't what we want. Let's say we start with the following Pod: + +{% highlight yaml %} +spec: + containers: + - name: nginx + image: nginx-1.0 +{% endhighlight %} + +...and we POST that to the server (as JSON). Then let's say we want to *add* a container to this Pod. + +{% highlight yaml %} +PATCH /api/v1/namespaces/default/pods/pod-name +spec: + containers: + - name: log-tailer + image: log-tailer-1.0 +{% endhighlight %} + +If we were to use standard Merge Patch, the entire container list would be replaced with the single log-tailer container. However, our intent is for the container lists to merge together based on the `name` field. + +To solve this problem, Strategic Merge Patch uses metadata attached to the API objects to determine what lists should be merged and which ones should not. Currently the metadata is available as struct tags on the API objects themselves, but will become available to clients as Swagger annotations in the future. In the above example, the `patchStrategy` metadata for the `containers` field would be `merge` and the `patchMergeKey` would be `name`. + +Note: If the patch results in merging two lists of scalars, the scalars are first deduplicated and then merged. + +Strategic Merge Patch also supports special operations as listed below. + +### List Operations + +To override the container list to be strictly replaced, regardless of the default: + +{% highlight yaml %} +containers: + - name: nginx + image: nginx-1.0 + - $patch: replace # any further $patch operations nested in this list will be ignored +{% endhighlight %} + +To delete an element of a list that should be merged: + +{% highlight yaml %} +containers: + - name: nginx + image: nginx-1.0 + - $patch: delete + name: log-tailer # merge key and value goes here +{% endhighlight %} + +### Map Operations + +To indicate that a map should not be merged and instead should be taken literally: + +{% highlight yaml %} +$patch: replace # recursive and applies to all fields of the map it's in +containers: +- name: nginx + image: nginx-1.0 +{% endhighlight %} + +To delete a field of a map: + +{% highlight yaml %} +name: nginx +image: nginx-1.0 +labels: + live: null # set the value of the map key to null +{% endhighlight %} + + +## Idempotency + +All compatible Kubernetes APIs MUST support "name idempotency" and respond with an HTTP status code 409 when a request is made to POST an object that has the same name as an existing object in the system. See [docs/user-guide/identifiers.md](../user-guide/identifiers) for details. + +Names generated by the system may be requested using `metadata.generateName`. GenerateName indicates that the name should be made unique by the server prior to persisting it. A non-empty value for the field indicates the name will be made unique (and the name returned to the client will be different than the name passed). The value of this field will be combined with a unique suffix on the server if the Name field has not been provided. The provided value must be valid within the rules for Name, and may be truncated by the length of the suffix required to make the value unique on the server. If this field is specified, and Name is not present, the server will NOT return a 409 if the generated name exists - instead, it will either return 201 Created or 504 with Reason `ServerTimeout` indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header). + +## Defaulting + +Default resource values are API version-specific, and they are applied during +the conversion from API-versioned declarative configuration to internal objects +representing the desired state (`Spec`) of the resource. Subsequent GETs of the +resource will include the default values explicitly. + +Incorporating the default values into the `Spec` ensures that `Spec` depicts the +full desired state so that it is easier for the system to determine how to +achieve the state, and for the user to know what to anticipate. + +API version-specific default values are set by the API server. + +## Late Initialization + +Late initialization is when resource fields are set by a system controller +after an object is created/updated. + +For example, the scheduler sets the `pod.spec.nodeName` field after the pod is created. + +Late-initializers should only make the following types of modifications: + - Setting previously unset fields + - Adding keys to maps + - Adding values to arrays which have mergeable semantics (`patchStrategy:"merge"` attribute in + the type definition). + +These conventions: + 1. allow a user (with sufficient privilege) to override any system-default behaviors by setting + the fields that would otherwise have been defaulted. + 1. enables updates from users to be merged with changes made during late initialization, using + strategic merge patch, as opposed to clobbering the change. + 1. allow the component which does the late-initialization to use strategic merge patch, which + facilitates composition and concurrency of such components. + +Although the apiserver Admission Control stage acts prior to object creation, +Admission Control plugins should follow the Late Initialization conventions +too, to allow their implementation to be later moved to a 'controller', or to client libraries. + +## Concurrency Control and Consistency + +Kubernetes leverages the concept of *resource versions* to achieve optimistic concurrency. All Kubernetes resources have a "resourceVersion" field as part of their metadata. This resourceVersion is a string that identifies the internal version of an object that can be used by clients to determine when objects have changed. When a record is about to be updated, it's version is checked against a pre-saved value, and if it doesn't match, the update fails with a StatusConflict (HTTP status code 409). + +The resourceVersion is changed by the server every time an object is modified. If resourceVersion is included with the PUT operation the system will verify that there have not been other successful mutations to the resource during a read/modify/write cycle, by verifying that the current value of resourceVersion matches the specified value. + +The resourceVersion is currently backed by [etcd's modifiedIndex](https://coreos.com/docs/distributed-configuration/etcd-api/). However, it's important to note that the application should *not* rely on the implementation details of the versioning system maintained by Kubernetes. We may change the implementation of resourceVersion in the future, such as to change it to a timestamp or per-object counter. + +The only way for a client to know the expected value of resourceVersion is to have received it from the server in response to a prior operation, typically a GET. This value MUST be treated as opaque by clients and passed unmodified back to the server. Clients should not assume that the resource version has meaning across namespaces, different kinds of resources, or different servers. Currently, the value of resourceVersion is set to match etcd's sequencer. You could think of it as a logical clock the API server can use to order requests. However, we expect the implementation of resourceVersion to change in the future, such as in the case we shard the state by kind and/or namespace, or port to another storage system. + +In the case of a conflict, the correct client action at this point is to GET the resource again, apply the changes afresh, and try submitting again. This mechanism can be used to prevent races like the following: + +``` +Client #1 Client #2 +GET Foo GET Foo +Set Foo.Bar = "one" Set Foo.Baz = "two" +PUT Foo PUT Foo +``` + +When these sequences occur in parallel, either the change to Foo.Bar or the change to Foo.Baz can be lost. + +On the other hand, when specifying the resourceVersion, one of the PUTs will fail, since whichever write succeeds changes the resourceVersion for Foo. + +resourceVersion may be used as a precondition for other operations (e.g., GET, DELETE) in the future, such as for read-after-write consistency in the presence of caching. + +"Watch" operations specify resourceVersion using a query parameter. It is used to specify the point at which to begin watching the specified resources. This may be used to ensure that no mutations are missed between a GET of a resource (or list of resources) and a subsequent Watch, even if the current version of the resource is more recent. This is currently the main reason that list operations (GET on a collection) return resourceVersion. + + +## Serialization Format + +APIs may return alternative representations of any resource in response to an Accept header or under alternative endpoints, but the default serialization for input and output of API responses MUST be JSON. + +All dates should be serialized as RFC3339 strings. + +## Units + +Units must either be explicit in the field name (e.g., `timeoutSeconds`), or must be specified as part of the value (e.g., `resource.Quantity`). Which approach is preferred is TBD, though currently we use the `fooSeconds` convention for durations. + + +## Selecting Fields + +Some APIs may need to identify which field in a JSON object is invalid, or to reference a value to extract from a separate resource. The current recommendation is to use standard JavaScript syntax for accessing that field, assuming the JSON object was transformed into a JavaScript object, without the leading dot, such as `metadata.name`. + +Examples: + +* Find the field "current" in the object "state" in the second item in the array "fields": `fields[1].state.current` + +## Object references + +Object references should either be called `fooName` if referring to an object of kind `Foo` by just the name (within the current namespace, if a namespaced resource), or should be called `fooRef`, and should contain a subset of the fields of the `ObjectReference` type. + + +TODO: Plugins, extensions, nested kinds, headers + + +## HTTP Status codes + +The server will respond with HTTP status codes that match the HTTP spec. See the section below for a breakdown of the types of status codes the server will send. + +The following HTTP status codes may be returned by the API. + +#### Success codes + +* `200 StatusOK` + * Indicates that the request completed successfully. +* `201 StatusCreated` + * Indicates that the request to create kind completed successfully. +* `204 StatusNoContent` + * Indicates that the request completed successfully, and the response contains no body. + * Returned in response to HTTP OPTIONS requests. + +#### Error codes + +* `307 StatusTemporaryRedirect` + * Indicates that the address for the requested resource has changed. + * Suggested client recovery behavior + * Follow the redirect. +* `400 StatusBadRequest` + * Indicates the requested is invalid. + * Suggested client recovery behavior: + * Do not retry. Fix the request. +* `401 StatusUnauthorized` + * Indicates that the server can be reached and understood the request, but refuses to take any further action, because the client must provide authorization. If the client has provided authorization, the server is indicating the provided authorization is unsuitable or invalid. + * Suggested client recovery behavior + * If the user has not supplied authorization information, prompt them for the appropriate credentials + * If the user has supplied authorization information, inform them their credentials were rejected and optionally prompt them again. +* `403 StatusForbidden` + * Indicates that the server can be reached and understood the request, but refuses to take any further action, because it is configured to deny access for some reason to the requested resource by the client. + * Suggested client recovery behavior + * Do not retry. Fix the request. +* `404 StatusNotFound` + * Indicates that the requested resource does not exist. + * Suggested client recovery behavior + * Do not retry. Fix the request. +* `405 StatusMethodNotAllowed` + * Indicates that the action the client attempted to perform on the resource was not supported by the code. + * Suggested client recovery behavior + * Do not retry. Fix the request. +* `409 StatusConflict` + * Indicates that either the resource the client attempted to create already exists or the requested update operation cannot be completed due to a conflict. + * Suggested client recovery behavior + * * If creating a new resource + * * Either change the identifier and try again, or GET and compare the fields in the pre-existing object and issue a PUT/update to modify the existing object. + * * If updating an existing resource: + * See `Conflict` from the `status` response section below on how to retrieve more information about the nature of the conflict. + * GET and compare the fields in the pre-existing object, merge changes (if still valid according to preconditions), and retry with the updated request (including `ResourceVersion`). +* `422 StatusUnprocessableEntity` + * Indicates that the requested create or update operation cannot be completed due to invalid data provided as part of the request. + * Suggested client recovery behavior + * Do not retry. Fix the request. +* `429 StatusTooManyRequests` + * Indicates that the either the client rate limit has been exceeded or the server has received more requests then it can process. + * Suggested client recovery behavior: + * Read the `Retry-After` HTTP header from the response, and wait at least that long before retrying. +* `500 StatusInternalServerError` + * Indicates that the server can be reached and understood the request, but either an unexpected internal error occurred and the outcome of the call is unknown, or the server cannot complete the action in a reasonable time (this maybe due to temporary server load or a transient communication issue with another server). + * Suggested client recovery behavior: + * Retry with exponential backoff. +* `503 StatusServiceUnavailable` + * Indicates that required service is unavailable. + * Suggested client recovery behavior: + * Retry with exponential backoff. +* `504 StatusServerTimeout` + * Indicates that the request could not be completed within the given time. Clients can get this response ONLY when they specified a timeout param in the request. + * Suggested client recovery behavior: + * Increase the value of the timeout param and retry with exponential backoff + +## Response Status Kind + +Kubernetes will always return the `Status` kind from any API endpoint when an error occurs. +Clients SHOULD handle these types of objects when appropriate. + +A `Status` kind will be returned by the API in two cases: + * When an operation is not successful (i.e. when the server would return a non 2xx HTTP status code). + * When a HTTP `DELETE` call is successful. + +The status object is encoded as JSON and provided as the body of the response. The status object contains fields for humans and machine consumers of the API to get more detailed information for the cause of the failure. The information in the status object supplements, but does not override, the HTTP status code's meaning. When fields in the status object have the same meaning as generally defined HTTP headers and that header is returned with the response, the header should be considered as having higher priority. + +**Example:** + +{% highlight console %} +$ curl -v -k -H "Authorization: Bearer WhCDvq4VPpYhrcfmF6ei7V9qlbqTubUc" https://10.240.122.184:443/api/v1/namespaces/default/pods/grafana + +> GET /api/v1/namespaces/default/pods/grafana HTTP/1.1 +> User-Agent: curl/7.26.0 +> Host: 10.240.122.184 +> Accept: */* +> Authorization: Bearer WhCDvq4VPpYhrcfmF6ei7V9qlbqTubUc +> + +< HTTP/1.1 404 Not Found +< Content-Type: application/json +< Date: Wed, 20 May 2015 18:10:42 GMT +< Content-Length: 232 +< +{ + "kind": "Status", + "apiVersion": "v1", + "metadata": {}, + "status": "Failure", + "message": "pods \"grafana\" not found", + "reason": "NotFound", + "details": { + "name": "grafana", + "kind": "pods" + }, + "code": 404 +} +{% endhighlight %} + +`status` field contains one of two possible values: +* `Success` +* `Failure` + +`message` may contain human-readable description of the error + +`reason` may contain a machine-readable, one-word, CamelCase description of why this operation is in the `Failure` status. If this value is empty there is no information available. The `reason` clarifies an HTTP status code but does not override it. + +`details` may contain extended data associated with the reason. Each reason may define its own extended details. This field is optional and the data returned is not guaranteed to conform to any schema except that defined by the reason type. + +Possible values for the `reason` and `details` fields: +* `BadRequest` + * Indicates that the request itself was invalid, because the request doesn't make any sense, for example deleting a read-only object. + * This is different than `status reason` `Invalid` above which indicates that the API call could possibly succeed, but the data was invalid. + * API calls that return BadRequest can never succeed. + * Http status code: `400 StatusBadRequest` +* `Unauthorized` + * Indicates that the server can be reached and understood the request, but refuses to take any further action without the client providing appropriate authorization. If the client has provided authorization, this error indicates the provided credentials are insufficient or invalid. + * Details (optional): + * `kind string` + * The kind attribute of the unauthorized resource (on some operations may differ from the requested resource). + * `name string` + * The identifier of the unauthorized resource. + * HTTP status code: `401 StatusUnauthorized` +* `Forbidden` + * Indicates that the server can be reached and understood the request, but refuses to take any further action, because it is configured to deny access for some reason to the requested resource by the client. + * Details (optional): + * `kind string` + * The kind attribute of the forbidden resource (on some operations may differ from the requested resource). + * `name string` + * The identifier of the forbidden resource. + * HTTP status code: `403 StatusForbidden` +* `NotFound` + * Indicates that one or more resources required for this operation could not be found. + * Details (optional): + * `kind string` + * The kind attribute of the missing resource (on some operations may differ from the requested resource). + * `name string` + * The identifier of the missing resource. + * HTTP status code: `404 StatusNotFound` +* `AlreadyExists` + * Indicates that the resource you are creating already exists. + * Details (optional): + * `kind string` + * The kind attribute of the conflicting resource. + * `name string` + * The identifier of the conflicting resource. + * HTTP status code: `409 StatusConflict` +* `Conflict` + * Indicates that the requested update operation cannot be completed due to a conflict. The client may need to alter the request. Each resource may define custom details that indicate the nature of the conflict. + * HTTP status code: `409 StatusConflict` +* `Invalid` + * Indicates that the requested create or update operation cannot be completed due to invalid data provided as part of the request. + * Details (optional): + * `kind string` + * the kind attribute of the invalid resource + * `name string` + * the identifier of the invalid resource + * `causes` + * One or more `StatusCause` entries indicating the data in the provided resource that was invalid. The `reason`, `message`, and `field` attributes will be set. + * HTTP status code: `422 StatusUnprocessableEntity` +* `Timeout` + * Indicates that the request could not be completed within the given time. Clients may receive this response if the server has decided to rate limit the client, or if the server is overloaded and cannot process the request at this time. + * Http status code: `429 TooManyRequests` + * The server should set the `Retry-After` HTTP header and return `retryAfterSeconds` in the details field of the object. A value of `0` is the default. +* `ServerTimeout` + * Indicates that the server can be reached and understood the request, but cannot complete the action in a reasonable time. This maybe due to temporary server load or a transient communication issue with another server. + * Details (optional): + * `kind string` + * The kind attribute of the resource being acted on. + * `name string` + * The operation that is being attempted. + * The server should set the `Retry-After` HTTP header and return `retryAfterSeconds` in the details field of the object. A value of `0` is the default. + * Http status code: `504 StatusServerTimeout` +* `MethodNotAllowed` + * Indicates that the action the client attempted to perform on the resource was not supported by the code. + * For instance, attempting to delete a resource that can only be created. + * API calls that return MethodNotAllowed can never succeed. + * Http status code: `405 StatusMethodNotAllowed` +* `InternalError` + * Indicates that an internal error occurred, it is unexpected and the outcome of the call is unknown. + * Details (optional): + * `causes` + * The original error. + * Http status code: `500 StatusInternalServerError` + +`code` may contain the suggested HTTP return code for this status. + + +## Events + +Events are complementary to status information, since they can provide some historical information about status and occurrences in addition to current or previous status. Generate events for situations users or administrators should be alerted about. + +Choose a unique, specific, short, CamelCase reason for each event category. For example, `FreeDiskSpaceInvalid` is a good event reason because it is likely to refer to just one situation, but `Started` is not a good reason because it doesn't sufficiently indicate what started, even when combined with other event fields. + +`Error creating foo` or `Error creating foo %s` would be appropriate for an event message, with the latter being preferable, since it is more informational. + +Accumulate repeated events in the client, especially for frequent events, to reduce data volume, load on the system, and noise exposed to users. + +## Naming conventions + +* Go field names must be CamelCase. JSON field names must be camelCase. Other than capitalization of the initial letter, the two should almost always match. No underscores nor dashes in either. +* Field and resource names should be declarative, not imperative (DoSomething, SomethingDoer, DoneBy, DoneAt). +* `Minion` has been deprecated in favor of `Node`. Use `Node` where referring to the node resource in the context of the cluster. Use `Host` where referring to properties of the individual physical/virtual system, such as `hostname`, `hostPath`, `hostNetwork`, etc. +* `FooController` is a deprecated kind naming convention. Name the kind after the thing being controlled instead (e.g., `Job` rather than `JobController`). +* The name of a field that specifies the time at which `something` occurs should be called `somethingTime`. Do not use `stamp` (e.g., `creationTimestamp`). +* We use the `fooSeconds` convention for durations, as discussed in the [units subsection](#units). + * `fooPeriodSeconds` is preferred for periodic intervals and other waiting periods (e.g., over `fooIntervalSeconds`). + * `fooTimeoutSeconds` is preferred for inactivity/unresponsiveness deadlines. + * `fooDeadlineSeconds` is preferred for activity completion deadlines. +* Do not use abbreviations in the API, except where they are extremely commonly used, such as "id", "args", or "stdin". +* Acronyms should similarly only be used when extremely commonly known. All letters in the acronym should have the same case, using the appropriate case for the situation. For example, at the beginning of a field name, the acronym should be all lowercase, such as "httpGet". Where used as a constant, all letters should be uppercase, such as "TCP" or "UDP". +* The name of a field referring to another resource of kind `Foo` by name should be called `fooName`. The name of a field referring to another resource of kind `Foo` by ObjectReference (or subset thereof) should be called `fooRef`. +* More generally, include the units and/or type in the field name if they could be ambiguous and they are not specified by the value or value type. + +## Label, selector, and annotation conventions + +Labels are the domain of users. They are intended to facilitate organization and management of API resources using attributes that are meaningful to users, as opposed to meaningful to the system. Think of them as user-created mp3 or email inbox labels, as opposed to the directory structure used by a program to store its data. The former is enables the user to apply an arbitrary ontology, whereas the latter is implementation-centric and inflexible. Users will use labels to select resources to operate on, display label values in CLI/UI columns, etc. Users should always retain full power and flexibility over the label schemas they apply to labels in their namespaces. + +However, we should support conveniences for common cases by default. For example, what we now do in ReplicationController is automatically set the RC's selector and labels to the labels in the pod template by default, if they are not already set. That ensures that the selector will match the template, and that the RC can be managed using the same labels as the pods it creates. Note that once we generalize selectors, it won't necessarily be possible to unambiguously generate labels that match an arbitrary selector. + +If the user wants to apply additional labels to the pods that it doesn't select upon, such as to facilitate adoption of pods or in the expectation that some label values will change, they can set the selector to a subset of the pod labels. Similarly, the RC's labels could be initialized to a subset of the pod template's labels, or could include additional/different labels. + +For disciplined users managing resources within their own namespaces, it's not that hard to consistently apply schemas that ensure uniqueness. One just needs to ensure that at least one value of some label key in common differs compared to all other comparable resources. We could/should provide a verification tool to check that. However, development of conventions similar to the examples in [Labels](../user-guide/labels) make uniqueness straightforward. Furthermore, relatively narrowly used namespaces (e.g., per environment, per application) can be used to reduce the set of resources that could potentially cause overlap. + +In cases where users could be running misc. examples with inconsistent schemas, or where tooling or components need to programmatically generate new objects to be selected, there needs to be a straightforward way to generate unique label sets. A simple way to ensure uniqueness of the set is to ensure uniqueness of a single label value, such as by using a resource name, uid, resource hash, or generation number. + +Problems with uids and hashes, however, include that they have no semantic meaning to the user, are not memorable nor readily recognizable, and are not predictable. Lack of predictability obstructs use cases such as creation of a replication controller from a pod, such as people want to do when exploring the system, bootstrapping a self-hosted cluster, or deletion and re-creation of a new RC that adopts the pods of the previous one, such as to rename it. Generation numbers are more predictable and much clearer, assuming there is a logical sequence. Fortunately, for deployments that's the case. For jobs, use of creation timestamps is common internally. Users should always be able to turn off auto-generation, in order to permit some of the scenarios described above. Note that auto-generated labels will also become one more field that needs to be stripped out when cloning a resource, within a namespace, in a new namespace, in a new cluster, etc., and will need to be ignored around when updating a resource via patch or read-modify-write sequence. + +Inclusion of a system prefix in a label key is fairly hostile to UX. A prefix is only necessary in the case that the user cannot choose the label key, in order to avoid collisions with user-defined labels. However, I firmly believe that the user should always be allowed to select the label keys to use on their resources, so it should always be possible to override default label keys. + +Therefore, resources supporting auto-generation of unique labels should have a `uniqueLabelKey` field, so that the user could specify the key if they wanted to, but if unspecified, it could be set by default, such as to the resource type, like job, deployment, or replicationController. The value would need to be at least spatially unique, and perhaps temporally unique in the case of job. + +Annotations have very different intended usage from labels. We expect them to be primarily generated and consumed by tooling and system extensions. I'm inclined to generalize annotations to permit them to directly store arbitrary json. Rigid names and name prefixes make sense, since they are analogous to API fields. + +In fact, in-development API fields, including those used to represent fields of newer alpha/beta API versions in the older stable storage version, may be represented as annotations with the form `something.alpha.kubernetes.io/name` or `something.beta.kubernetes.io/name` (depending on our confidence in it). For example `net.alpha.kubernetes.io/policy` might represent an experimental network policy field. + +Other advice regarding use of labels, annotations, and other generic map keys by Kubernetes components and tools: + - Key names should be all lowercase, with words separated by dashes, such as `desired-replicas` + - Prefix the key with `kubernetes.io/` or `foo.kubernetes.io/`, preferably the latter if the label/annotation is specific to `foo` + - For instance, prefer `service-account.kubernetes.io/name` over `kubernetes.io/service-account.name` + - Use annotations to store API extensions that the controller responsible for the resource doesn't need to know about, experimental fields that aren't intended to be generally used API fields, etc. Beware that annotations aren't automatically handled by the API conversion machinery. + + + + diff --git a/v1.1/docs/devel/api_changes.md b/v1.1/docs/devel/api_changes.md index 9757d8b041..f3ac8b27e5 100644 --- a/v1.1/docs/devel/api_changes.md +++ b/v1.1/docs/devel/api_changes.md @@ -1,13 +1,9 @@ --- title: "So you want to change the API?" --- - - -# So you want to change the API? - Before attempting a change to the API, you should familiarize yourself with a number of existing API types and with the [API -conventions](api-conventions.html). If creating a new API +conventions](api-conventions). If creating a new API type/resource, we also recommend that you first send a PR containing just a proposal for the new API types, and that you initially target the extensions API (pkg/apis/extensions). @@ -96,27 +92,27 @@ Let's consider some examples. In a hypothetical API (assume we're at version v6), the `Frobber` struct looks something like this: {% highlight go %} -{% raw %} + // API v6. type Frobber struct { Height int `json:"height"` Param string `json:"param"` } -{% endraw %} + {% endhighlight %} You want to add a new `Width` field. It is generally safe to add new fields without changing the API version, so you can simply change it to: {% highlight go %} -{% raw %} + // Still API v6. type Frobber struct { Height int `json:"height"` Width int `json:"width"` Param string `json:"param"` } -{% endraw %} + {% endhighlight %} The onus is on you to define a sane default value for `Width` such that rule #1 @@ -128,7 +124,7 @@ simply change `Param string` to `Params []string` (without creating a whole new API version) - that fails rules #1 and #2. You can instead do something like: {% highlight go %} -{% raw %} + // Still API v6, but kind of clumsy. type Frobber struct { Height int `json:"height"` @@ -136,7 +132,7 @@ type Frobber struct { Param string `json:"param"` // the first param ExtraParams []string `json:"params"` // additional params } -{% endraw %} + {% endhighlight %} Now you can satisfy the rules: API calls that provide the old style `Param` @@ -148,14 +144,14 @@ distinct from any one version is to handle growth like this. The internal representation can be implemented as: {% highlight go %} -{% raw %} + // Internal, soon to be v7beta1. type Frobber struct { Height int Width int Params []string } -{% endraw %} + {% endhighlight %} The code that converts to/from versioned APIs can decode this into the somewhat @@ -179,14 +175,14 @@ you add units to `height` and `width`. You implement this by adding duplicate fields: {% highlight go %} -{% raw %} + type Frobber struct { Height *int `json:"height"` Width *int `json:"width"` HeightInInches *int `json:"heightInInches"` WidthInInches *int `json:"widthInInches"` } -{% endraw %} + {% endhighlight %} You convert all of the fields to pointers in order to distinguish between unset and @@ -202,38 +198,38 @@ in the case of an old client that was only aware of the old field (e.g., `height Say the client creates: {% highlight json %} -{% raw %} + { "height": 10, "width": 5 } -{% endraw %} + {% endhighlight %} and GETs: {% highlight json %} -{% raw %} + { "height": 10, "heightInInches": 10, "width": 5, "widthInInches": 5 } -{% endraw %} + {% endhighlight %} then PUTs back: {% highlight json %} -{% raw %} + { "height": 13, "heightInInches": 10, "width": 5, "widthInInches": 5 } -{% endraw %} + {% endhighlight %} The update should not fail, because it would have worked before `heightInInches` was added. @@ -270,8 +266,8 @@ Breaking compatibility of a beta or stable API version, such as v1, is unaccepta Compatibility for experimental or alpha APIs is not strictly required, but breaking compatibility should not be done lightly, as it disrupts all users of the feature. Experimental APIs may be removed. Alpha and beta API versions may be deprecated -and eventually removed wholesale, as described in the [versioning document](../design/versioning.html). -Document incompatible changes across API versions under the [conversion tips](../api.html). +and eventually removed wholesale, as described in the [versioning document](../design/versioning). +Document incompatible changes across API versions under the [conversion tips](../api). If your change is going to be backward incompatible or might be a breaking change for API consumers, please send an announcement to `kubernetes-dev@googlegroups.com` before @@ -405,9 +401,9 @@ regenerate auto-generated ones. To regenerate them: - run {% highlight sh %} -{% raw %} + hack/update-generated-conversions.sh -{% endraw %} + {% endhighlight %} If running the above script is impossible due to compile errors, the easiest @@ -433,9 +429,9 @@ To regenerate them: - run {% highlight sh %} -{% raw %} + hack/update-generated-deep-copies.sh -{% endraw %} + {% endhighlight %} ## Edit json (un)marshaling code @@ -451,9 +447,9 @@ To regenerate them: - run {% highlight sh %} -{% raw %} + hack/update-codecgen.sh -{% endraw %} + {% endhighlight %} ## Making a new API Group @@ -518,7 +514,7 @@ doing! ## Write end-to-end tests -Check out the [E2E docs](e2e-tests.html) for detailed information about how to write end-to-end +Check out the [E2E docs](e2e-tests) for detailed information about how to write end-to-end tests for your feature. ## Examples and docs @@ -536,9 +532,9 @@ an example to illustrate your change. Make sure you update the swagger API spec by running: {% highlight sh %} -{% raw %} + hack/update-swagger-spec.sh -{% endraw %} + {% endhighlight %} The API spec changes should be in a commit separate from your other changes. @@ -590,7 +586,7 @@ New feature development proceeds through a series of stages of increasing maturi upgrade may require downtime for anything relying on the new feature, and may require manual conversion of objects to the new version; when manual conversion is necessary, the project will provide documentation on the process (for an example, see [v1 conversion - tips](../api.html)) + tips](../api)) - Cluster Reliability: since the feature has e2e tests, enabling the feature via a flag should not create new bugs in unrelated features; because the feature is new, it may have minor bugs - Support: the project commits to complete the feature, in some form, in a subsequent Stable diff --git a/v1.1/docs/devel/automation.md b/v1.1/docs/devel/automation.md index e0bec999b0..beae8a15fd 100644 --- a/v1.1/docs/devel/automation.md +++ b/v1.1/docs/devel/automation.md @@ -1,10 +1,6 @@ --- title: "Kubernetes Development Automation" --- - - -# Kubernetes Development Automation - ## Overview Kubernetes uses a variety of automated tools in an attempt to relieve developers of repeptitive, low @@ -24,13 +20,13 @@ for kubernetes. The submit-queue does the following: {% highlight go %} -{% raw %} + for _, pr := range readyToMergePRs() { if testsAreStable() { mergePR(pr) } } -{% endraw %} + {% endhighlight %} The status of the submit-queue is [online.](http://submit-queue.k8s.io/) diff --git a/v1.1/docs/devel/cherry-picks.md b/v1.1/docs/devel/cherry-picks.md index e0200587c7..c865df3d05 100644 --- a/v1.1/docs/devel/cherry-picks.md +++ b/v1.1/docs/devel/cherry-picks.md @@ -1,10 +1,6 @@ --- title: "Overview" --- - - -# Overview - This document explains cherry picks are managed on release branches within the Kubernetes projects. @@ -13,9 +9,9 @@ Kubernetes projects. Any contributor can propose a cherry pick of any pull request, like so: {% highlight sh %} -{% raw %} + hack/cherry_pick_pull.sh upstream/release-3.14 98765 -{% endraw %} + {% endhighlight %} This will walk you through the steps to propose an automated cherry pick of pull diff --git a/v1.1/docs/devel/cli-roadmap.md b/v1.1/docs/devel/cli-roadmap.md index 1103a18edb..9fb632ac25 100644 --- a/v1.1/docs/devel/cli-roadmap.md +++ b/v1.1/docs/devel/cli-roadmap.md @@ -1,10 +1,6 @@ --- title: "Kubernetes CLI/Configuration Roadmap" --- - - -# Kubernetes CLI/Configuration Roadmap - See github issues with the following labels: * [area/app-config-deployment](https://github.com/kubernetes/kubernetes/labels/area/app-config-deployment) * [component/kubectl](https://github.com/kubernetes/kubernetes/labels/component/kubectl) diff --git a/v1.1/docs/devel/client-libraries.md b/v1.1/docs/devel/client-libraries.md index c442ffba93..850bdec276 100644 --- a/v1.1/docs/devel/client-libraries.md +++ b/v1.1/docs/devel/client-libraries.md @@ -1,10 +1,6 @@ --- title: "Kubernetes API client libraries" --- - - -## Kubernetes API client libraries - ### Supported * [Go](http://releases.k8s.io/release-1.1/pkg/client/) diff --git a/v1.1/docs/devel/coding-conventions.md b/v1.1/docs/devel/coding-conventions.md index bcc003e049..f6434310eb 100644 --- a/v1.1/docs/devel/coding-conventions.md +++ b/v1.1/docs/devel/coding-conventions.md @@ -9,7 +9,7 @@ Code conventions - Go - Ensure your code passes the [presubmit checks](development.html#hooks) - [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments) - - [Effective Go](https://golang.org/doc/effective_go.html) + - [Effective Go](https://golang.org/doc/effective_go) - Comment your code. - [Go's commenting conventions](http://blog.golang.org/godoc-documenting-go-code) - If reviewers ask questions about why the code is the way it is, that's a sign that comments might be helpful. @@ -24,10 +24,10 @@ Code conventions - Importers can use a different name if they need to disambiguate. - Locks should be called `lock` and should never be embedded (always `lock sync.Mutex`). When multiple locks are present, give each lock a distinct name following Go conventions - `stateLock`, `mapLock` etc. - API conventions - - [API changes](api_changes.html) - - [API conventions](api-conventions.html) - - [Kubectl conventions](kubectl-conventions.html) - - [Logging conventions](logging.html) + - [API changes](api_changes) + - [API conventions](api-conventions) + - [Kubectl conventions](kubectl-conventions) + - [Logging conventions](logging) Testing conventions - All new packages and most new significant functionality must come with unit tests @@ -44,7 +44,7 @@ Directory and file conventions - Package directories should generally avoid using separators as much as possible (when packages are multiple words, they usually should be in nested subdirectories). - Document directories and filenames should use dashes rather than underscores - Contrived examples that illustrate system features belong in /docs/user-guide or /docs/admin, depending on whether it is a feature primarily intended for users that deploy applications or cluster administrators, respectively. Actual application examples belong in /examples. - - Examples should also illustrate [best practices for using the system](../user-guide/config-best-practices.html) + - Examples should also illustrate [best practices for using the system](../user-guide/config-best-practices) - Third-party code - Third-party Go code is managed using Godeps - Other third-party code belongs in /third_party diff --git a/v1.1/docs/devel/collab.md b/v1.1/docs/devel/collab.md index 8c78052564..1a00723f3d 100644 --- a/v1.1/docs/devel/collab.md +++ b/v1.1/docs/devel/collab.md @@ -1,10 +1,6 @@ --- title: "On Collaborative Development" --- - - -# On Collaborative Development - Kubernetes is open source, but many of the people working on it do so as their day job. In order to avoid forcing people to be "at work" effectively 24/7, we want to establish some semi-formal protocols around development. Hopefully these rules make things go more smoothly. If you find that this is not the case, please complain loudly. ## Patches welcome diff --git a/v1.1/docs/devel/developer-guides/vagrant.md b/v1.1/docs/devel/developer-guides/vagrant.md index d01d610b5c..081a8965e4 100644 --- a/v1.1/docs/devel/developer-guides/vagrant.md +++ b/v1.1/docs/devel/developer-guides/vagrant.md @@ -1,10 +1,6 @@ --- title: "Getting started with Vagrant" --- - - -## Getting started with Vagrant - Running kubernetes with Vagrant (and VirtualBox) is an easy way to run/test/develop on your local machine (Linux, Mac OS X). ### Prerequisites @@ -15,31 +11,31 @@ Running kubernetes with Vagrant (and VirtualBox) is an easy way to run/test/deve 2. [VMWare Fusion](https://www.vmware.com/products/fusion/) version 5 or greater as well as the appropriate [Vagrant VMWare Fusion provider](https://www.vagrantup.com/vmware) 3. [VMWare Workstation](https://www.vmware.com/products/workstation/) version 9 or greater as well as the [Vagrant VMWare Workstation provider](https://www.vagrantup.com/vmware) 4. [Parallels Desktop](https://www.parallels.com/products/desktop/) version 9 or greater as well as the [Vagrant Parallels provider](https://parallels.github.io/vagrant-parallels/) -3. Get or build a [binary release](../../../docs/getting-started-guides/binary_release.html) +3. Get or build a [binary release](/{{page.version}}/docs/getting-started-guides/binary_release) ### Setup By default, the Vagrant setup will create a single master VM (called kubernetes-master) and one node (called kubernetes-minion-1). Each VM will take 1 GB, so make sure you have at least 2GB to 4GB of free memory (plus appropriate free disk space). To start your local cluster, open a shell and run: {% highlight sh %} -{% raw %} + cd kubernetes export KUBERNETES_PROVIDER=vagrant ./cluster/kube-up.sh -{% endraw %} + {% endhighlight %} The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster management scripts which variant to use. If you forget to set this, the assumption is you are running on Google Compute Engine. -If you installed more than one Vagrant provider, Kubernetes will usually pick the appropriate one. However, you can override which one Kubernetes will use by setting the [`VAGRANT_DEFAULT_PROVIDER`](https://docs.vagrantup.com/v2/providers/default.html) environment variable: +If you installed more than one Vagrant provider, Kubernetes will usually pick the appropriate one. However, you can override which one Kubernetes will use by setting the [`VAGRANT_DEFAULT_PROVIDER`](https://docs.vagrantup.com/v2/providers/default) environment variable: {% highlight sh %} -{% raw %} + export VAGRANT_DEFAULT_PROVIDER=parallels export KUBERNETES_PROVIDER=vagrant ./cluster/kube-up.sh -{% endraw %} + {% endhighlight %} Vagrant will provision each machine in the cluster with all the necessary components to run Kubernetes. The initial setup can take a few minutes to complete on each machine. @@ -49,25 +45,25 @@ By default, each VM in the cluster is running Fedora, and all of the Kubernetes To access the master or any node: {% highlight sh %} -{% raw %} + vagrant ssh master vagrant ssh minion-1 -{% endraw %} + {% endhighlight %} If you are running more than one nodes, you can access the others by: {% highlight sh %} -{% raw %} + vagrant ssh minion-2 vagrant ssh minion-3 -{% endraw %} + {% endhighlight %} To view the service status and/or logs on the kubernetes-master: {% highlight console %} -{% raw %} + $ vagrant ssh master [vagrant@kubernetes-master ~] $ sudo systemctl status kube-apiserver [vagrant@kubernetes-master ~] $ sudo journalctl -r -u kube-apiserver @@ -77,19 +73,19 @@ $ vagrant ssh master [vagrant@kubernetes-master ~] $ sudo systemctl status etcd [vagrant@kubernetes-master ~] $ sudo systemctl status nginx -{% endraw %} + {% endhighlight %} To view the services on any of the nodes: {% highlight console %} -{% raw %} + $ vagrant ssh minion-1 [vagrant@kubernetes-minion-1] $ sudo systemctl status docker [vagrant@kubernetes-minion-1] $ sudo journalctl -r -u docker [vagrant@kubernetes-minion-1] $ sudo systemctl status kubelet [vagrant@kubernetes-minion-1] $ sudo journalctl -r -u kubelet -{% endraw %} + {% endhighlight %} ### Interacting with your Kubernetes cluster with Vagrant. @@ -99,26 +95,26 @@ With your Kubernetes cluster up, you can manage the nodes in your cluster with t To push updates to new Kubernetes code after making source changes: {% highlight sh %} -{% raw %} + ./cluster/kube-push.sh -{% endraw %} + {% endhighlight %} To stop and then restart the cluster: {% highlight sh %} -{% raw %} + vagrant halt ./cluster/kube-up.sh -{% endraw %} + {% endhighlight %} To destroy the cluster: {% highlight sh %} -{% raw %} + vagrant destroy -{% endraw %} + {% endhighlight %} Once your Vagrant machines are up and provisioned, the first thing to do is to check that you can use the `kubectl.sh` script. @@ -126,14 +122,14 @@ Once your Vagrant machines are up and provisioned, the first thing to do is to c You may need to build the binaries first, you can do this with `make` {% highlight console %} -{% raw %} + $ ./cluster/kubectl.sh get nodes NAME LABELS STATUS kubernetes-minion-0whl kubernetes.io/hostname=kubernetes-minion-0whl Ready kubernetes-minion-4jdf kubernetes.io/hostname=kubernetes-minion-4jdf Ready kubernetes-minion-epbe kubernetes.io/hostname=kubernetes-minion-epbe Ready -{% endraw %} + {% endhighlight %} ### Interacting with your Kubernetes cluster with the `kube-*` scripts. @@ -143,41 +139,41 @@ Alternatively to using the vagrant commands, you can also use the `cluster/kube- All of these commands assume you have set `KUBERNETES_PROVIDER` appropriately: {% highlight sh %} -{% raw %} + export KUBERNETES_PROVIDER=vagrant -{% endraw %} + {% endhighlight %} Bring up a vagrant cluster {% highlight sh %} -{% raw %} + ./cluster/kube-up.sh -{% endraw %} + {% endhighlight %} Destroy the vagrant cluster {% highlight sh %} -{% raw %} + ./cluster/kube-down.sh -{% endraw %} + {% endhighlight %} Update the vagrant cluster after you make changes (only works when building your own releases locally): {% highlight sh %} -{% raw %} + ./cluster/kube-push.sh -{% endraw %} + {% endhighlight %} Interact with the cluster {% highlight sh %} -{% raw %} + ./cluster/kubectl.sh -{% endraw %} + {% endhighlight %} ### Authenticating with your master @@ -185,7 +181,7 @@ Interact with the cluster When using the vagrant provider in Kubernetes, the `cluster/kubectl.sh` script will cache your credentials in a `~/.kubernetes_vagrant_auth` file so you will not be prompted for them in the future. {% highlight console %} -{% raw %} + $ cat ~/.kubernetes_vagrant_auth { "User": "vagrant", "Password": "vagrant" @@ -193,15 +189,15 @@ $ cat ~/.kubernetes_vagrant_auth "CertFile": "/home/k8s_user/.kubecfg.vagrant.crt", "KeyFile": "/home/k8s_user/.kubecfg.vagrant.key" } -{% endraw %} + {% endhighlight %} You should now be set to use the `cluster/kubectl.sh` script. For example try to list the nodes that you have started with: {% highlight sh %} -{% raw %} + ./cluster/kubectl.sh get nodes -{% endraw %} + {% endhighlight %} ### Running containers @@ -209,14 +205,14 @@ You should now be set to use the `cluster/kubectl.sh` script. For example try to Your cluster is running, you can list the nodes in your cluster: {% highlight console %} -{% raw %} + $ ./cluster/kubectl.sh get nodes NAME LABELS STATUS kubernetes-minion-0whl kubernetes.io/hostname=kubernetes-minion-0whl Ready kubernetes-minion-4jdf kubernetes.io/hostname=kubernetes-minion-4jdf Ready kubernetes-minion-epbe kubernetes.io/hostname=kubernetes-minion-epbe Ready -{% endraw %} + {% endhighlight %} Now start running some containers! @@ -225,7 +221,7 @@ You can now use any of the cluster/kube-*.sh commands to interact with your VM m Before starting a container there will be no pods, services and replication controllers. {% highlight console %} -{% raw %} + $ cluster/kubectl.sh get pods NAME READY STATUS RESTARTS AGE @@ -234,59 +230,59 @@ NAME LABELS SELECTOR IP(S) PORT(S) $ cluster/kubectl.sh get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -{% endraw %} + {% endhighlight %} Start a container running nginx with a replication controller and three replicas {% highlight console %} -{% raw %} + $ cluster/kubectl.sh run my-nginx --image=nginx --replicas=3 --port=80 CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS my-nginx my-nginx nginx run=my-nginx 3 -{% endraw %} + {% endhighlight %} When listing the pods, you will see that three containers have been started and are in Waiting state: {% highlight console %} -{% raw %} + $ cluster/kubectl.sh get pods NAME READY STATUS RESTARTS AGE my-nginx-389da 1/1 Waiting 0 33s my-nginx-kqdjk 1/1 Waiting 0 33s my-nginx-nyj3x 1/1 Waiting 0 33s -{% endraw %} + {% endhighlight %} You need to wait for the provisioning to complete, you can monitor the minions by doing: {% highlight console %} -{% raw %} + $ sudo salt '*minion-1' cmd.run 'docker images' kubernetes-minion-1: REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE 96864a7d2df3 26 hours ago 204.4 MB kubernetes/pause latest 6c4579af347b 8 weeks ago 239.8 kB -{% endraw %} + {% endhighlight %} Once the docker image for nginx has been downloaded, the container will start and you can list it: {% highlight console %} -{% raw %} + $ sudo salt '*minion-1' cmd.run 'docker ps' kubernetes-minion-1: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES dbe79bf6e25b nginx:latest "nginx" 21 seconds ago Up 19 seconds k8s--mynginx.8c5b8a3a--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--fcfa837f fa0e29c94501 kubernetes/pause:latest "/pause" 8 minutes ago Up 8 minutes 0.0.0.0:8080->80/tcp k8s--net.a90e7ce4--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--baf5b21b -{% endraw %} + {% endhighlight %} Going back to listing the pods, services and replicationcontrollers, you now have: {% highlight console %} -{% raw %} + $ cluster/kubectl.sh get pods NAME READY STATUS RESTARTS AGE my-nginx-389da 1/1 Running 0 33s @@ -299,21 +295,21 @@ NAME LABELS SELECTOR IP(S) PORT(S) $ cluster/kubectl.sh get rc NAME IMAGE(S) SELECTOR REPLICAS my-nginx nginx run=my-nginx 3 -{% endraw %} + {% endhighlight %} We did not start any services, hence there are none listed. But we see three replicas displayed properly. -Check the [guestbook](../../../examples/guestbook/README.html) application to learn how to create a service. +Check the [guestbook](../../../examples/guestbook/README) application to learn how to create a service. You can already play with scaling the replicas with: {% highlight console %} -{% raw %} + $ ./cluster/kubectl.sh scale rc my-nginx --replicas=2 $ ./cluster/kubectl.sh get pods NAME READY STATUS RESTARTS AGE my-nginx-kqdjk 1/1 Running 0 13m my-nginx-nyj3x 1/1 Running 0 13m -{% endraw %} + {% endhighlight %} Congratulations! @@ -323,9 +319,9 @@ Congratulations! The following will run all of the end-to-end testing scenarios assuming you set your environment in `cluster/kube-env.sh`: {% highlight sh %} -{% raw %} + NUM_MINIONS=3 hack/e2e-test.sh -{% endraw %} + {% endhighlight %} ### Troubleshooting @@ -335,12 +331,12 @@ NUM_MINIONS=3 hack/e2e-test.sh By default the Vagrantfile will download the box from S3. You can change this (and cache the box locally) by providing a name and an alternate URL when calling `kube-up.sh` {% highlight sh %} -{% raw %} + export KUBERNETES_BOX_NAME=choose_your_own_name_for_your_kuber_box export KUBERNETES_BOX_URL=path_of_your_kuber_box export KUBERNETES_PROVIDER=vagrant ./cluster/kube-up.sh -{% endraw %} + {% endhighlight %} #### I just created the cluster, but I am getting authorization errors! @@ -348,21 +344,21 @@ export KUBERNETES_PROVIDER=vagrant You probably have an incorrect ~/.kubernetes_vagrant_auth file for the cluster you are attempting to contact. {% highlight sh %} -{% raw %} + rm ~/.kubernetes_vagrant_auth -{% endraw %} + {% endhighlight %} After using kubectl.sh make sure that the correct credentials are set: {% highlight console %} -{% raw %} + $ cat ~/.kubernetes_vagrant_auth { "User": "vagrant", "Password": "vagrant" } -{% endraw %} + {% endhighlight %} #### I just created the cluster, but I do not see my container running! @@ -383,9 +379,9 @@ Are you sure you built a release first? Did you install `net-tools`? For more cl You can control the number of nodes that are instantiated via the environment variable `NUM_MINIONS` on your host machine. If you plan to work with replicas, we strongly encourage you to work with enough nodes to satisfy your largest intended replica size. If you do not plan to work with replicas, you can save some system resources by running with a single node. You do this, by setting `NUM_MINIONS` to 1 like so: {% highlight sh %} -{% raw %} + export NUM_MINIONS=1 -{% endraw %} + {% endhighlight %} #### I want my VMs to have more memory! @@ -394,18 +390,18 @@ You can control the memory allotted to virtual machines with the `KUBERNETES_MEM Just set it to the number of megabytes you would like the machines to have. For example: {% highlight sh %} -{% raw %} + export KUBERNETES_MEMORY=2048 -{% endraw %} + {% endhighlight %} If you need more granular control, you can set the amount of memory for the master and nodes independently. For example: {% highlight sh %} -{% raw %} + export KUBERNETES_MASTER_MEMORY=1536 export KUBERNETES_MINION_MEMORY=2048 -{% endraw %} + {% endhighlight %} #### I ran vagrant suspend and nothing works! diff --git a/v1.1/docs/devel/development.md b/v1.1/docs/devel/development.md index bcd4fcd507..8c0a954b3b 100644 --- a/v1.1/docs/devel/development.md +++ b/v1.1/docs/devel/development.md @@ -1,17 +1,13 @@ --- title: "Development Guide" --- - - -# Development Guide - # Releases and Official Builds Official releases are built in Docker containers. Details are [here](http://releases.k8s.io/release-1.1/build/README.md). You can do simple builds and development with just a local Docker installation. If want to build go locally outside of docker, please continue below. ## Go development environment -Kubernetes is written in [Go](http://golang.org) programming language. If you haven't set up Go development environment, please follow [this instruction](http://golang.org/doc/code.html) to install go tool and set up GOPATH. Ensure your version of Go is at least 1.3. +Kubernetes is written in [Go](http://golang.org) programming language. If you haven't set up Go development environment, please follow [this instruction](http://golang.org/doc/code) to install go tool and set up GOPATH. Ensure your version of Go is at least 1.3. ## Git Setup @@ -31,56 +27,56 @@ Below, we outline one of the more common git workflows that core developers use. The commands below require that you have $GOPATH set ([$GOPATH docs](https://golang.org/doc/code.html#GOPATH)). We highly recommend you put Kubernetes' code into your GOPATH. Note: the commands below will not work if there is more than one directory in your `$GOPATH`. {% highlight sh %} -{% raw %} + mkdir -p $GOPATH/src/k8s.io cd $GOPATH/src/k8s.io # Replace "$YOUR_GITHUB_USERNAME" below with your github username git clone https://github.com/$YOUR_GITHUB_USERNAME/kubernetes.git cd kubernetes git remote add upstream 'https://github.com/kubernetes/kubernetes.git' -{% endraw %} + {% endhighlight %} ### Create a branch and make changes {% highlight sh %} -{% raw %} + git checkout -b myfeature # Make your code changes -{% endraw %} + {% endhighlight %} ### Keeping your development fork in sync {% highlight sh %} -{% raw %} + git fetch upstream git rebase upstream/master -{% endraw %} + {% endhighlight %} Note: If you have write access to the main repository at github.com/kubernetes/kubernetes, you should modify your git configuration so that you can't accidentally push to upstream: {% highlight sh %} -{% raw %} + git remote set-url --push upstream no_push -{% endraw %} + {% endhighlight %} ### Committing changes to your fork {% highlight sh %} -{% raw %} + git commit git push -f origin myfeature -{% endraw %} + {% endhighlight %} ### Creating a pull request 1. Visit https://github.com/$YOUR_GITHUB_USERNAME/kubernetes 2. Click the "Compare and pull request" button next to your "myfeature" branch. -3. Check out the pull request [process](pull-requests.html) for more details +3. Check out the pull request [process](pull-requests) for more details ### When to retain commits and when to squash @@ -94,7 +90,7 @@ fixups (e.g. automated doc formatting), use one or more commits for the changes to tooling and a final commit to apply the fixup en masse. This makes reviews much easier. -See [Faster Reviews](faster_reviews.html) for more details. +See [Faster Reviews](faster_reviews) for more details. ## godep and dependency management @@ -111,20 +107,20 @@ directly from mercurial. 2) Create a new GOPATH for your tools and install godep: {% highlight sh %} -{% raw %} + export GOPATH=$HOME/go-tools mkdir -p $GOPATH go get github.com/tools/godep -{% endraw %} + {% endhighlight %} 3) Add $GOPATH/bin to your path. Typically you'd add this to your ~/.profile: {% highlight sh %} -{% raw %} + export GOPATH=$HOME/go-tools export PATH=$PATH:$GOPATH/bin -{% endraw %} + {% endhighlight %} ### Using godep @@ -136,40 +132,40 @@ Here's a quick walkthrough of one way to use godeps to add or update a Kubernete _Devoting a separate directory is not required, but it is helpful to separate dependency updates from other changes._ {% highlight sh %} -{% raw %} + export KPATH=$HOME/code/kubernetes mkdir -p $KPATH/src/k8s.io/kubernetes cd $KPATH/src/k8s.io/kubernetes git clone https://path/to/your/fork . # Or copy your existing local repo here. IMPORTANT: making a symlink doesn't work. -{% endraw %} + {% endhighlight %} 2) Set up your GOPATH. {% highlight sh %} -{% raw %} + # Option A: this will let your builds see packages that exist elsewhere on your system. export GOPATH=$KPATH:$GOPATH # Option B: This will *not* let your local builds see packages that exist elsewhere on your system. export GOPATH=$KPATH # Option B is recommended if you're going to mess with the dependencies. -{% endraw %} + {% endhighlight %} 3) Populate your new GOPATH. {% highlight sh %} -{% raw %} + cd $KPATH/src/k8s.io/kubernetes godep restore -{% endraw %} + {% endhighlight %} 4) Next, you can either add a new dependency or update an existing one. {% highlight sh %} -{% raw %} + # To add a new dependency, do: cd $KPATH/src/k8s.io/kubernetes go get path/to/dependency @@ -181,7 +177,7 @@ cd $KPATH/src/k8s.io/kubernetes go get -u path/to/dependency # Change code in Kubernetes accordingly if necessary. godep update path/to/dependency/... -{% endraw %} + {% endhighlight %} _If `go get -u path/to/dependency` fails with compilation errors, instead try `go get -d -u path/to/dependency` @@ -203,41 +199,41 @@ Before committing any changes, please link/copy these hooks into your .git directory. This will keep you from accidentally committing non-gofmt'd go code. {% highlight sh %} -{% raw %} + cd kubernetes/.git/hooks/ ln -s ../../hooks/pre-commit . -{% endraw %} + {% endhighlight %} ## Unit tests {% highlight sh %} -{% raw %} + cd kubernetes hack/test-go.sh -{% endraw %} + {% endhighlight %} Alternatively, you could also run: {% highlight sh %} -{% raw %} + cd kubernetes godep go test ./... -{% endraw %} + {% endhighlight %} If you only want to run unit tests in one package, you could run ``godep go test`` under the package directory. For example, the following commands will run all unit tests in package kubelet: {% highlight console %} -{% raw %} + $ cd kubernetes # step into the kubernetes directory. $ cd pkg/kubelet $ godep go test # some output from unit tests PASS ok k8s.io/kubernetes/pkg/kubelet 0.317s -{% endraw %} + {% endhighlight %} ## Coverage @@ -247,10 +243,10 @@ Currently, collecting coverage is only supported for the Go unit tests. To run all unit tests and generate an HTML coverage report, run the following: {% highlight sh %} -{% raw %} + cd kubernetes KUBE_COVER=y hack/test-go.sh -{% endraw %} + {% endhighlight %} At the end of the run, an the HTML report will be generated with the path printed to stdout. @@ -258,10 +254,10 @@ At the end of the run, an the HTML report will be generated with the path printe To run tests and collect coverage in only one package, pass its relative path under the `kubernetes` directory as an argument, for example: {% highlight sh %} -{% raw %} + cd kubernetes KUBE_COVER=y hack/test-go.sh pkg/kubectl -{% endraw %} + {% endhighlight %} Multiple arguments can be passed, in which case the coverage results will be combined for all tests run. @@ -273,10 +269,10 @@ Coverage results for the project can also be viewed on [Coveralls](https://cover You need an [etcd](https://github.com/coreos/etcd/releases/tag/v2.0.0) in your path, please make sure it is installed and in your ``$PATH``. {% highlight sh %} -{% raw %} + cd kubernetes hack/test-integration.sh -{% endraw %} + {% endhighlight %} ## End-to-End tests @@ -284,18 +280,18 @@ hack/test-integration.sh You can run an end-to-end test which will bring up a master and two nodes, perform some tests, and then tear everything down. Make sure you have followed the getting started steps for your chosen cloud platform (which might involve changing the `KUBERNETES_PROVIDER` environment variable to something other than "gce". {% highlight sh %} -{% raw %} + cd kubernetes hack/e2e-test.sh -{% endraw %} + {% endhighlight %} Pressing control-C should result in an orderly shutdown but if something goes wrong and you still have some VMs running you can force a cleanup with this command: {% highlight sh %} -{% raw %} + go run hack/e2e.go --down -{% endraw %} + {% endhighlight %} ### Flag options @@ -303,7 +299,7 @@ go run hack/e2e.go --down See the flag definitions in `hack/e2e.go` for more options, such as reusing an existing cluster, here is an overview: {% highlight sh %} -{% raw %} + # Build binaries for testing go run hack/e2e.go --build @@ -330,13 +326,13 @@ go run hack/e2e.go -v -test --test_args="--ginkgo.focus=Pods.*env" # Alternately, if you have the e2e cluster up and no desire to see the event stream, you can run ginkgo-e2e.sh directly: hack/ginkgo-e2e.sh --ginkgo.focus=Pods.*env -{% endraw %} + {% endhighlight %} ### Combining flags {% highlight sh %} -{% raw %} + # Flags can be combined, and their actions will take place in this order: # -build, -push|-up|-pushup, -test|-tests=..., -down # e.g.: @@ -350,14 +346,14 @@ go run hack/e2e.go -build -pushup -test -down # kubectl output. go run hack/e2e.go -v -ctl='get events' go run hack/e2e.go -v -ctl='delete pod foobar' -{% endraw %} + {% endhighlight %} ## Conformance testing End-to-end testing, as described above, is for [development -distributions](writing-a-getting-started-guide.html). A conformance test is used on -a [versioned distro](writing-a-getting-started-guide.html). +distributions](writing-a-getting-started-guide). A conformance test is used on +a [versioned distro](writing-a-getting-started-guide). The conformance test runs a subset of the e2e-tests against a manually-created cluster. It does not require support for up/push/down and other operations. To run a conformance test, you need to know the @@ -367,14 +363,14 @@ See [conformance-test.sh](http://releases.k8s.io/release-1.1/hack/conformance-te ## Testing out flaky tests -[Instructions here](flaky-tests.html) +[Instructions here](flaky-tests) ## Regenerating the CLI documentation {% highlight sh %} -{% raw %} + hack/update-generated-docs.sh -{% endraw %} + {% endhighlight %} diff --git a/v1.1/docs/devel/e2e-tests.md b/v1.1/docs/devel/e2e-tests.md index 4563444ad3..9fcbacbed3 100644 --- a/v1.1/docs/devel/e2e-tests.md +++ b/v1.1/docs/devel/e2e-tests.md @@ -1,10 +1,6 @@ --- title: "End-2-End Testing in Kubernetes" --- - - -# End-2-End Testing in Kubernetes - ## Overview The end-2-end tests for kubernetes provide a mechanism to test behavior of the system, and to ensure end user operations match developer specifications. In distributed systems it is not uncommon that a minor change may pass all unit tests, but cause unforseen changes at the system level. Thus, the primary objectives of the end-2-end tests are to ensure a consistent and reliable behavior of the kubernetes code base, and to catch bugs early. @@ -32,7 +28,7 @@ The output for the end-2-end tests will be a single binary called `e2e.test` und For the purposes of brevity, we will look at a subset of the options, which are listed below: ``` -{% raw %} + -ginkgo.dryRun=false: If set, ginkgo will walk the test hierarchy without actually running anything. Best paired with -v. -ginkgo.failFast=false: If set, ginkgo will stop running a test suite after a failure occurs. -ginkgo.failOnPending=false: If set, ginkgo will mark the test suite as failed if any specs are pending. @@ -45,18 +41,18 @@ For the purposes of brevity, we will look at a subset of the options, which are -prom-push-gateway="": The URL to prometheus gateway, so that metrics can be pushed during e2es and scraped by prometheus. Typically something like 127.0.0.1:9091. -provider="": The name of the Kubernetes provider (gce, gke, local, vagrant, etc.) -repo-root="../../": Root directory of kubernetes repository, for finding test files. -{% endraw %} + ``` Prior to running the tests, it is recommended that you first create a simple auth file in your home directory, e.g. `$HOME/.kubernetes_auth` , with the following: ``` -{% raw %} + { "User": "root", "Password": "" } -{% endraw %} + ``` Next, you will need a cluster that you can test against. As mentioned earlier, you will want to execute `sudo ./hack/local-up-cluster.sh`. To get a sense of what tests exist, you may want to run: @@ -89,14 +85,14 @@ If a behavior does not currently have coverage and a developer wishes to add a n Another benefit of the end-2-end tests is the ability to create reproducible loads on the system, which can then be used to determine the responsiveness, or analyze other characteristics of the system. For example, the density tests load the system to 30,50,100 pods per/node and measures the different characteristics of the system, such as throughput, api-latency, etc. -For a good overview of how we analyze performance data, please read the following [post](http://blog.kubernetes.io/2015/09/kubernetes-performance-measurements-and.html) +For a good overview of how we analyze performance data, please read the following [post](http://blog.kubernetes.io/2015/09/kubernetes-performance-measurements-and) For developers who are interested in doing their own performance analysis, we recommend setting up [prometheus](http://prometheus.io/) for data collection, and using [promdash](http://prometheus.io/docs/visualization/promdash/) to visualize the data. There also exists the option of pushing your own metrics in from the tests using a [prom-push-gateway](http://prometheus.io/docs/instrumenting/pushing/). Containers for all of these components can be found [here](https://hub.docker.com/u/prom/). For more accurate measurements, you may wish to set up prometheus external to kubernetes in an environment where it can access the major system components (api-server, controller-manager, scheduler). This is especially useful when attempting to gather metrics in a load-balanced api-server environment, because all api-servers can be analyzed independently as well as collectively. On startup, configuration file is passed to prometheus that specifies the endpoints that prometheus will scrape, as well as the sampling interval. ``` -{% raw %} + #prometheus.conf job: { name: "kubernetes" @@ -109,7 +105,7 @@ job: { # controller-manager target: "http://localhost:10252/metrics" } -{% endraw %} + ``` Once prometheus is scraping the kubernetes endpoints, that data can then be plotted using promdash, and alerts can be created against the assortment of metrics that kubernetes provides. diff --git a/v1.1/docs/devel/faster_reviews.md b/v1.1/docs/devel/faster_reviews.md index ffeb7d7d4c..dd4dcb38b7 100644 --- a/v1.1/docs/devel/faster_reviews.md +++ b/v1.1/docs/devel/faster_reviews.md @@ -1,10 +1,6 @@ --- title: "How to get faster PR reviews" --- - - -# How to get faster PR reviews - Most of what is written here is not at all specific to Kubernetes, but it bears being written down in the hope that it will occasionally remind people of "best practices" around code reviews. @@ -27,10 +23,10 @@ Let's talk about how to avoid this. ## 0. Familiarize yourself with project conventions -* [Development guide](development.html) -* [Coding conventions](coding-conventions.html) -* [API conventions](api-conventions.html) -* [Kubectl conventions](kubectl-conventions.html) +* [Development guide](development) +* [Coding conventions](coding-conventions) +* [API conventions](api-conventions) +* [Kubectl conventions](kubectl-conventions) ## 1. Don't build a cathedral in one PR diff --git a/v1.1/docs/devel/flaky-tests.md b/v1.1/docs/devel/flaky-tests.md index f8baa5dd16..bfbe77b1ca 100644 --- a/v1.1/docs/devel/flaky-tests.md +++ b/v1.1/docs/devel/flaky-tests.md @@ -1,10 +1,6 @@ --- title: "Hunting flaky tests in Kubernetes" --- - - -# Hunting flaky tests in Kubernetes - Sometimes unit tests are flaky. This means that due to (usually) race conditions, they will occasionally fail, even though most of the time they pass. We have a goal of 99.9% flake free tests. This means that there is only one flake in one thousand runs of a test. @@ -18,7 +14,7 @@ There is a testing image `brendanburns/flake` up on the docker hub. We will use Create a replication controller with the following config: {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: ReplicationController metadata: @@ -38,15 +34,15 @@ spec: value: pkg/tools - name: REPO_SPEC value: https://github.com/kubernetes/kubernetes -{% endraw %} + {% endhighlight %} Note that we omit the labels and the selector fields of the replication controller, because they will be populated from the labels field of the pod template by default. {% highlight sh %} -{% raw %} + kubectl create -f ./controller.yaml -{% endraw %} + {% endhighlight %} This will spin up 24 instances of the test. They will run to completion, then exit, and the kubelet will restart them, accumulating more and more runs of the test. @@ -54,7 +50,7 @@ You can examine the recent runs of the test by calling `docker ps -a` and lookin You can use this script to automate checking for failures, assuming your cluster is running on GCE and has four nodes: {% highlight sh %} -{% raw %} + echo "" > output.txt for i in {1..4}; do echo "Checking kubernetes-minion-${i}" @@ -62,15 +58,15 @@ for i in {1..4}; do gcloud compute ssh "kubernetes-minion-${i}" --command="sudo docker ps -a" >> output.txt done grep "Exited ([^0])" output.txt -{% endraw %} + {% endhighlight %} Eventually you will have sufficient runs for your purposes. At that point you can stop and delete the replication controller by running: {% highlight sh %} -{% raw %} + kubectl stop replicationcontroller flakecontroller -{% endraw %} + {% endhighlight %} If you do a final check for flakes with `docker ps -a`, ignore tasks that exited -1, since that's what happens when you stop the replication controller. diff --git a/v1.1/docs/devel/getting-builds.md b/v1.1/docs/devel/getting-builds.md index 8cdbd23a65..0af090b080 100644 --- a/v1.1/docs/devel/getting-builds.md +++ b/v1.1/docs/devel/getting-builds.md @@ -1,10 +1,6 @@ --- title: "Getting Kubernetes Builds" --- - - -# Getting Kubernetes Builds - You can use [hack/get-build.sh](http://releases.k8s.io/release-1.1/hack/get-build.sh) to or use as a reference on how to get the most recent builds with curl. With `get-build.sh` you can grab the most recent stable build, the most recent release candidate, or the most recent build to pass our ci and gce e2e tests (essentially a nightly build). Run `./hack/get-build.sh -h` for its usage. @@ -12,36 +8,36 @@ Run `./hack/get-build.sh -h` for its usage. For example, to get a build at a specific version (v1.0.2): {% highlight console %} -{% raw %} + ./hack/get-build.sh v1.0.2 -{% endraw %} + {% endhighlight %} Alternatively, to get the latest stable release: {% highlight console %} -{% raw %} + ./hack/get-build.sh release/stable -{% endraw %} + {% endhighlight %} Finally, you can just print the latest or stable version: {% highlight console %} -{% raw %} + ./hack/get-build.sh -v ci/latest -{% endraw %} + {% endhighlight %} You can also use the gsutil tool to explore the Google Cloud Storage release buckets. Here are some examples: {% highlight sh %} -{% raw %} + gsutil cat gs://kubernetes-release/ci/latest.txt # output the latest ci version number gsutil cat gs://kubernetes-release/ci/latest-green.txt # output the latest ci version number that passed gce e2e gsutil ls gs://kubernetes-release/ci/v0.20.0-29-g29a55cc/ # list the contents of a ci release gsutil ls gs://kubernetes-release/release # list all official releases and rcs -{% endraw %} + {% endhighlight %} diff --git a/v1.1/docs/devel/index.md b/v1.1/docs/devel/index.md index ae73742b50..564ce00a81 100644 --- a/v1.1/docs/devel/index.md +++ b/v1.1/docs/devel/index.md @@ -1,47 +1,43 @@ --- title: "Kubernetes Developer Guide" --- - - -# Kubernetes Developer Guide - The developer guide is for anyone wanting to either write code which directly accesses the Kubernetes API, or to contribute directly to the Kubernetes project. -It assumes some familiarity with concepts in the [User Guide](../user-guide/README.html) and the [Cluster Admin -Guide](../admin/README.html). +It assumes some familiarity with concepts in the [User Guide](../user-guide/README) and the [Cluster Admin +Guide](../admin/README). ## The process of developing and contributing code to the Kubernetes project -* **On Collaborative Development** ([collab.md](collab.html)): Info on pull requests and code reviews. +* **On Collaborative Development** ([collab.md](collab)): Info on pull requests and code reviews. -* **GitHub Issues** ([issues.md](issues.html)): How incoming issues are reviewed and prioritized. +* **GitHub Issues** ([issues.md](issues)): How incoming issues are reviewed and prioritized. -* **Pull Request Process** ([pull-requests.md](pull-requests.html)): When and why pull requests are closed. +* **Pull Request Process** ([pull-requests.md](pull-requests)): When and why pull requests are closed. -* **Faster PR reviews** ([faster_reviews.md](faster_reviews.html)): How to get faster PR reviews. +* **Faster PR reviews** ([faster_reviews.md](faster_reviews)): How to get faster PR reviews. -* **Getting Recent Builds** ([getting-builds.md](getting-builds.html)): How to get recent builds including the latest builds that pass CI. +* **Getting Recent Builds** ([getting-builds.md](getting-builds)): How to get recent builds including the latest builds that pass CI. -* **Automated Tools** ([automation.md](automation.html)): Descriptions of the automation that is running on our github repository. +* **Automated Tools** ([automation.md](automation)): Descriptions of the automation that is running on our github repository. ## Setting up your dev environment, coding, and debugging -* **Development Guide** ([development.md](development.html)): Setting up your development environment. +* **Development Guide** ([development.md](development)): Setting up your development environment. -* **Hunting flaky tests** ([flaky-tests.md](flaky-tests.html)): We have a goal of 99.9% flake free tests. +* **Hunting flaky tests** ([flaky-tests.md](flaky-tests)): We have a goal of 99.9% flake free tests. Here's how to run your tests many times. -* **Logging Conventions** ([logging.md](logging.html)]: Glog levels. +* **Logging Conventions** ([logging.md](logging)]: Glog levels. -* **Profiling Kubernetes** ([profiling.md](profiling.html)): How to plug in go pprof profiler to Kubernetes. +* **Profiling Kubernetes** ([profiling.md](profiling)): How to plug in go pprof profiler to Kubernetes. * **Instrumenting Kubernetes with a new metric** - ([instrumentation.md](instrumentation.html)): How to add a new metrics to the + ([instrumentation.md](instrumentation)): How to add a new metrics to the Kubernetes code base. -* **Coding Conventions** ([coding-conventions.md](coding-conventions.html)): +* **Coding Conventions** ([coding-conventions.md](coding-conventions)): Coding style advice for contributors. @@ -49,33 +45,33 @@ Guide](../admin/README.html). * API objects are explained at [http://kubernetes.io/third_party/swagger-ui/](http://kubernetes.io/third_party/swagger-ui/). -* **Annotations** ([docs/user-guide/annotations.md](../user-guide/annotations.html)): are for attaching arbitrary non-identifying metadata to objects. +* **Annotations** ([docs/user-guide/annotations.md](../user-guide/annotations)): are for attaching arbitrary non-identifying metadata to objects. Programs that automate Kubernetes objects may use annotations to store small amounts of their state. -* **API Conventions** ([api-conventions.md](api-conventions.html)): +* **API Conventions** ([api-conventions.md](api-conventions)): Defining the verbs and resources used in the Kubernetes API. -* **API Client Libraries** ([client-libraries.md](client-libraries.html)): +* **API Client Libraries** ([client-libraries.md](client-libraries)): A list of existing client libraries, both supported and user-contributed. ## Writing plugins -* **Authentication Plugins** ([docs/admin/authentication.md](../admin/authentication.html)): +* **Authentication Plugins** ([docs/admin/authentication.md](../admin/authentication)): The current and planned states of authentication tokens. -* **Authorization Plugins** ([docs/admin/authorization.md](../admin/authorization.html)): +* **Authorization Plugins** ([docs/admin/authorization.md](../admin/authorization)): Authorization applies to all HTTP requests on the main apiserver port. This doc explains the available authorization implementations. -* **Admission Control Plugins** ([admission_control](../design/admission_control.html)) +* **Admission Control Plugins** ([admission_control](../design/admission_control)) ## Building releases -* **Making release notes** ([making-release-notes.md](making-release-notes.html)): Generating release nodes for a new release. +* **Making release notes** ([making-release-notes.md](making-release-notes)): Generating release nodes for a new release. -* **Releasing Kubernetes** ([releasing.md](releasing.html)): How to create a Kubernetes release (as in version) +* **Releasing Kubernetes** ([releasing.md](releasing)): How to create a Kubernetes release (as in version) and how the version information gets embedded into the built binaries. diff --git a/v1.1/docs/devel/kubectl-conventions.md b/v1.1/docs/devel/kubectl-conventions.md index 6f49527758..5b3e9369ed 100644 --- a/v1.1/docs/devel/kubectl-conventions.md +++ b/v1.1/docs/devel/kubectl-conventions.md @@ -8,16 +8,8 @@ Kubectl Conventions Updated: 8/27/2015 -**Table of Contents** - +{% include pagetoc.html %} - - [Principles](#principles) - - [Command conventions](#command-conventions) - - [Flag conventions](#flag-conventions) - - [Output conventions](#output-conventions) - - [Documentation conventions](#documentation-conventions) - - ## Principles diff --git a/v1.1/docs/devel/making-release-notes.md b/v1.1/docs/devel/making-release-notes.md index d93006e929..01ec5cfc57 100644 --- a/v1.1/docs/devel/making-release-notes.md +++ b/v1.1/docs/devel/making-release-notes.md @@ -1,10 +1,6 @@ --- title: "Making release notes" --- - - -## Making release notes - This documents the process for making release notes for a release. ### 1) Note the PR number of the previous release @@ -17,9 +13,9 @@ Find the most-recent PR that was merged with the current .0 release. Remember t ### 2) Run the release-notes tool {% highlight bash %} -{% raw %} + ${KUBERNETES_ROOT}/build/make-release-notes.sh $LASTPR $CURRENTPR -{% endraw %} + {% endhighlight %} ### 3) Trim the release notes diff --git a/v1.1/docs/devel/profiling.md b/v1.1/docs/devel/profiling.md index 2c0152481d..d185a02471 100644 --- a/v1.1/docs/devel/profiling.md +++ b/v1.1/docs/devel/profiling.md @@ -1,10 +1,6 @@ --- title: "Profiling Kubernetes" --- - - -# Profiling Kubernetes - This document explain how to plug in profiler and how to profile Kubernetes services. ## Profiling library @@ -16,11 +12,11 @@ Go comes with inbuilt 'net/http/pprof' profiling library and profiling web servi TL;DR: Add lines: {% highlight go %} -{% raw %} + m.mux.HandleFunc("/debug/pprof/", pprof.Index) m.mux.HandleFunc("/debug/pprof/profile", pprof.Profile) m.mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol) -{% endraw %} + {% endhighlight %} to the init(c *Config) method in 'pkg/master/master.go' and import 'net/http/pprof' package. @@ -32,17 +28,17 @@ In most use cases to use profiler service it's enough to do 'import _ net/http/p Even when running profiler I found not really straightforward to use 'go tool pprof' with it. The problem is that at least for dev purposes certificates generated for APIserver are not signed by anyone trusted and because secureServer serves only secure traffic it isn't straightforward to connect to the service. The best workaround I found is by creating an ssh tunnel from the kubernetes_master open unsecured port to some external server, and use this server as a proxy. To save everyone looking for correct ssh flags, it is done by running: {% highlight sh %} -{% raw %} + ssh kubernetes_master -L:localhost:8080 -{% endraw %} + {% endhighlight %} or analogous one for you Cloud provider. Afterwards you can e.g. run {% highlight sh %} -{% raw %} + go tool pprof http://localhost:/debug/pprof/profile -{% endraw %} + {% endhighlight %} to get 30 sec. CPU profile. diff --git a/v1.1/docs/devel/pull-requests.md b/v1.1/docs/devel/pull-requests.md index 4fd2ec501f..2d83db67e1 100644 --- a/v1.1/docs/devel/pull-requests.md +++ b/v1.1/docs/devel/pull-requests.md @@ -37,7 +37,7 @@ Automation ---------- We use a variety of automation to manage pull requests. This automation is described in detail -[elsewhere.](automation.html) +[elsewhere.](automation) diff --git a/v1.1/docs/devel/releasing.md b/v1.1/docs/devel/releasing.md index 9f5dd168fa..c71bc93876 100644 --- a/v1.1/docs/devel/releasing.md +++ b/v1.1/docs/devel/releasing.md @@ -1,10 +1,6 @@ --- title: "Releasing Kubernetes" --- - - -# Releasing Kubernetes - This document explains how to cut a release, and the theory behind it. If you just want to cut a release and move on with your life, you can stop reading after the first section. @@ -38,9 +34,9 @@ can find the Git hash for a build by looking at the "Console Log", then look for `githash=`. You should see a line line: {% highlight console %} -{% raw %} + + githash=v0.20.2-322-g974377b -{% endraw %} + {% endhighlight %} Because Jenkins builds frequently, if you're looking between jobs @@ -55,9 +51,9 @@ oncall. Before proceeding to the next step: {% highlight sh %} -{% raw %} + export BRANCHPOINT=v0.20.2-322-g974377b -{% endraw %} + {% endhighlight %} Where `v0.20.2-322-g974377b` is the git hash you decided on. This will become @@ -95,7 +91,7 @@ In your git repo (you still have `${VER}` set from above right?): #### Writing Release Notes -[This helpful guide](making-release-notes.html) describes how to write release +[This helpful guide](making-release-notes) describes how to write release notes for a major/minor release. In the release template on GitHub, leave the last PR number that the tool finds for the `.0` release, so the next releaser doesn't have to hunt. @@ -107,7 +103,7 @@ doesn't have to hunt. We cut `vX.Y.Z` releases from the `release-vX.Y` branch after all cherry picks to the branch have been resolved. You should ensure all outstanding cherry picks have been reviewed and merged and the branch validated on Jenkins (validation -TBD). See the [Cherry Picks](cherry-picks.html) for more information on how to +TBD). See the [Cherry Picks](cherry-picks) for more information on how to manage cherry picks prior to cutting the release. #### Tagging and Merging @@ -207,12 +203,12 @@ We are using `pkg/version/base.go` as the source of versioning in absence of information from git. Here is a sample of that file's contents: {% highlight go %} -{% raw %} + var ( gitVersion string = "v0.4-dev" // version from git, output of $(git describe) gitCommit string = "" // sha1 from git, output of $(git rev-parse HEAD) ) -{% endraw %} + {% endhighlight %} This means a build with `go install` or `go get` or a build from a tarball will @@ -292,7 +288,7 @@ As an example, Docker commit a327d9b91edf has a `v1.1.1-N-gXXX` label but it is not present in Docker `v1.2.0`: {% highlight console %} -{% raw %} + $ git describe a327d9b91edf v1.1.1-822-ga327d9b91edf @@ -300,7 +296,7 @@ $ git log --oneline v1.2.0..a327d9b91edf a327d9b91edf Fix data space reporting from Kb/Mb to KB/MB (Non-empty output here means the commit is not present on v1.2.0.) -{% endraw %} + {% endhighlight %} ## Release Notes diff --git a/v1.1/docs/devel/scheduler.md b/v1.1/docs/devel/scheduler.md index 7e57ff5dbc..4da88461d8 100755 --- a/v1.1/docs/devel/scheduler.md +++ b/v1.1/docs/devel/scheduler.md @@ -1,10 +1,6 @@ --- title: "The Kubernetes Scheduler" --- - - -# The Kubernetes Scheduler - The Kubernetes scheduler runs as a process alongside the other master components such as the API server. Its interface to the API server is to watch for Pods with an empty PodSpec.NodeName, and for each Pod, it posts a Binding diff --git a/v1.1/docs/devel/scheduler_algorithm.md b/v1.1/docs/devel/scheduler_algorithm.md index 5fe029ef30..47d51144ea 100755 --- a/v1.1/docs/devel/scheduler_algorithm.md +++ b/v1.1/docs/devel/scheduler_algorithm.md @@ -1,18 +1,14 @@ --- title: "Scheduler Algorithm in Kubernetes" --- - - -# Scheduler Algorithm in Kubernetes - -For each unscheduled Pod, the Kubernetes scheduler tries to find a node across the cluster according to a set of rules. A general introduction to the Kubernetes scheduler can be found at [scheduler.md](scheduler.html). In this document, the algorithm of how to select a node for the Pod is explained. There are two steps before a destination node of a Pod is chosen. The first step is filtering all the nodes and the second is ranking the remaining nodes to find a best fit for the Pod. +For each unscheduled Pod, the Kubernetes scheduler tries to find a node across the cluster according to a set of rules. A general introduction to the Kubernetes scheduler can be found at [scheduler.md](scheduler). In this document, the algorithm of how to select a node for the Pod is explained. There are two steps before a destination node of a Pod is chosen. The first step is filtering all the nodes and the second is ranking the remaining nodes to find a best fit for the Pod. ## Filtering the nodes The purpose of filtering the nodes is to filter out the nodes that do not meet certain requirements of the Pod. For example, if the free resource on a node (measured by the capacity minus the sum of the resource requests of all the Pods that already run on the node) is less than the Pod's required resource, the node should not be considered in the ranking phase so it is filtered out. Currently, there are several "predicates" implementing different filtering policies, including: - `NoDiskConflict`: Evaluate if a pod can fit due to the volumes it requests, and those that are already mounted. -- `PodFitsResources`: Check if the free resource (CPU and Memory) meets the requirement of the Pod. The free resource is measured by the capacity minus the sum of requests of all Pods on the node. To learn more about the resource QoS in Kubernetes, please check [QoS proposal](../proposals/resource-qos.html). +- `PodFitsResources`: Check if the free resource (CPU and Memory) meets the requirement of the Pod. The free resource is measured by the capacity minus the sum of requests of all Pods on the node. To learn more about the resource QoS in Kubernetes, please check [QoS proposal](../proposals/resource-qos). - `PodFitsHostPorts`: Check if any HostPort required by the Pod is already occupied on the node. - `PodFitsHost`: Filter out all nodes except the one specified in the PodSpec's NodeName field. - `PodSelectorMatches`: Check if the labels of the node match the labels specified in the Pod's `nodeSelector` field ([Here](../user-guide/node-selection/) is an example of how to use `nodeSelector` field). @@ -36,7 +32,7 @@ Currently, Kubernetes scheduler provides some practical priority functions, incl - `CalculateSpreadPriority`: Spread Pods by minimizing the number of Pods belonging to the same service on the same node. - `CalculateAntiAffinityPriority`: Spread Pods by minimizing the number of Pods belonging to the same service on nodes with the same value for a particular label. -The details of the above priority functions can be found in [plugin/pkg/scheduler/algorithm/priorities](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithm/priorities/). Kubernetes uses some, but not all, of these priority functions by default. You can see which ones are used by default in [plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go). Similar as predicates, you can combine the above priority functions and assign weight factors (positive number) to them as you want (check [scheduler.md](scheduler.html) for how to customize). +The details of the above priority functions can be found in [plugin/pkg/scheduler/algorithm/priorities](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithm/priorities/). Kubernetes uses some, but not all, of these priority functions by default. You can see which ones are used by default in [plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go). Similar as predicates, you can combine the above priority functions and assign weight factors (positive number) to them as you want (check [scheduler.md](scheduler) for how to customize). diff --git a/v1.1/docs/devel/writing-a-getting-started-guide.md b/v1.1/docs/devel/writing-a-getting-started-guide.md index ecfba702ce..7f9c3c8a93 100644 --- a/v1.1/docs/devel/writing-a-getting-started-guide.md +++ b/v1.1/docs/devel/writing-a-getting-started-guide.md @@ -1,10 +1,6 @@ --- title: "Writing a Getting Started Guide" --- - - -# Writing a Getting Started Guide - This page gives some advice for anyone planning to write or update a Getting Started Guide for Kubernetes. It also gives some guidelines which reviewers should follow when reviewing a pull request for a guide. @@ -16,7 +12,7 @@ A guide refers to scripts, Configuration Management files, and/or binary assets the combination of all these things needed to run on a particular type of infrastructure a **distro**. -[The Matrix](../../docs/getting-started-guides/README.html) lists the distros. If there is already a guide +[The Matrix](/{{page.version}}/docs/getting-started-guides/README) lists the distros. If there is already a guide which is similar to the one you have planned, consider improving that one. @@ -37,7 +33,7 @@ These guidelines say *what* to do. See the Rationale section for *why*. search for uses of flags by guides. - We may ask that you host binary assets or large amounts of code in our `contrib` directory or on your own repo. - - Add or update a row in [The Matrix](../../docs/getting-started-guides/README.html). + - Add or update a row in [The Matrix](/{{page.version}}/docs/getting-started-guides/README). - State the binary version of Kubernetes that you tested clearly in your Guide doc. - Setup a cluster and run the [conformance test](development.html#conformance-testing) against it, and report the results in your PR. diff --git a/v1.1/docs/getting-started-guides/README.md b/v1.1/docs/getting-started-guides/README.md index 095257df4d..babc2ad16b 100644 --- a/v1.1/docs/getting-started-guides/README.md +++ b/v1.1/docs/getting-started-guides/README.md @@ -1,18 +1,15 @@ --- title: "Picking the Right Solution" -section: guides --- Kubernetes can run on a range of platforms, from your laptop, to VMs on a cloud provider, to rack of bare metal servers. The effort required to set up a cluster varies from running a single command to crafting your own customized cluster. We'll guide you in picking a solution that fits for your needs. -## Table of Contents - {% include pagetoc.html %} ## Options -If you just want to "kick the tires" on Kubernetes, we recommend the [local Docker-based](docker.html) solution. +If you just want to "kick the tires" on Kubernetes, we recommend the [local Docker-based](docker) solution. The local Docker-based solution is one of several [Local cluster](#local-machine-solutions) solutions that are quick to set up, but are limited to running on one machine. @@ -35,9 +32,9 @@ But their size and availability is limited to that of a single machine. The local-machine solutions are: -- [Local Docker-based](docker.html) (recommended starting point) -- [Vagrant](vagrant.html) (works on any platform with Vagrant: Linux, MacOS, or Windows.) -- [No-VM local cluster](locally.html) (Linux only) +- [Local Docker-based](docker) (recommended starting point) +- [Vagrant](vagrant) (works on any platform with Vagrant: Linux, MacOS, or Windows.) +- [No-VM local cluster](locally) (Linux only) ### Hosted Solutions @@ -50,9 +47,9 @@ clusters. These solutions allow you to create Kubernetes clusters on a range of Cloud IaaS providers with only a few commands, and have active community support. -- [GCE](gce.html) -- [AWS](aws.html) -- [Azure](coreos/azure/README.html) +- [GCE](gce) +- [AWS](aws) +- [Azure](/{{page.version}}/docs/getting-started-guides/coreos/azure/README) ### Custom Solutions @@ -62,50 +59,50 @@ base operating systems. If you can find a guide below that matches your needs, use it. It may be a little out of date, but it will be easier than starting from scratch. If you do want to start from scratch because you have special requirements or just because you want to understand what is underneath a Kubernetes -cluster, try the [Getting Started from Scratch](scratch.html) guide. +cluster, try the [Getting Started from Scratch](scratch) guide. If you are interested in supporting Kubernetes on a new platform, check out our [advice for -writing a new solution](../../docs/devel/writing-a-getting-started-guide.html). +writing a new solution](/{{page.version}}/docs/devel/writing-a-getting-started-guide). #### Cloud These solutions are combinations of cloud provider and OS not covered by the above solutions. -- [AWS + coreos](coreos.html) -- [GCE + CoreOS](coreos.html) -- [AWS + Ubuntu](juju.html) -- [Joyent + Ubuntu](juju.html) -- [Rackspace + CoreOS](rackspace.html) +- [AWS + coreos](coreos) +- [GCE + CoreOS](coreos) +- [AWS + Ubuntu](juju) +- [Joyent + Ubuntu](juju) +- [Rackspace + CoreOS](rackspace) #### On-Premises VMs -- [Vagrant](coreos.html) (uses CoreOS and flannel) -- [CloudStack](cloudstack.html) (uses Ansible, CoreOS and flannel) -- [Vmware](vsphere.html) (uses Debian) -- [juju.md](juju.html) (uses Juju, Ubuntu and flannel) -- [Vmware](coreos.html) (uses CoreOS and flannel) -- [libvirt-coreos.md](libvirt-coreos.html) (uses CoreOS) -- [oVirt](ovirt.html) -- [libvirt](fedora/flannel_multi_node_cluster.html) (uses Fedora and flannel) -- [KVM](fedora/flannel_multi_node_cluster.html) (uses Fedora and flannel) +- [Vagrant](coreos) (uses CoreOS and flannel) +- [CloudStack](cloudstack) (uses Ansible, CoreOS and flannel) +- [Vmware](vsphere) (uses Debian) +- [juju.md](juju) (uses Juju, Ubuntu and flannel) +- [Vmware](coreos) (uses CoreOS and flannel) +- [libvirt-coreos.md](libvirt-coreos) (uses CoreOS) +- [oVirt](ovirt) +- [libvirt](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) (uses Fedora and flannel) +- [KVM](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) (uses Fedora and flannel) #### Bare Metal -- [Offline](coreos/bare_metal_offline.html) (no internet required. Uses CoreOS and Flannel) -- [fedora/fedora_ansible_config.md](fedora/fedora_ansible_config.html) -- [Fedora single node](fedora/fedora_manual_config.html) -- [Fedora multi node](fedora/flannel_multi_node_cluster.html) -- [Centos](centos/centos_manual_config.html) -- [Ubuntu](ubuntu.html) -- [Docker Multi Node](docker-multinode.html) +- [Offline](/{{page.version}}/docs/getting-started-guides/coreos/bare_metal_offline) (no internet required. Uses CoreOS and Flannel) +- [fedora/fedora_ansible_config.md](/{{page.version}}/docs/getting-started-guides/fedora/fedora_ansible_config) +- [Fedora single node](/{{page.version}}/docs/getting-started-guides/fedora/fedora_manual_config) +- [Fedora multi node](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) +- [Centos](/{{page.version}}/docs/getting-started-guides/centos/centos_manual_config) +- [Ubuntu](ubuntu) +- [Docker Multi Node](docker-multinode) #### Integrations These solutions provide integration with 3rd party schedulers, resource managers, and/or lower level platforms. -- [Kubernetes on Mesos](mesos.html) +- [Kubernetes on Mesos](mesos) - Instructions specify GCE, but are generic enough to be adapted to most existing Mesos clusters -- [Kubernetes on DCOS](dcos.html) +- [Kubernetes on DCOS](dcos) - Community Edition DCOS uses AWS - Enterprise Edition DCOS supports cloud hosting, on-premise VMs, and bare metal @@ -115,40 +112,40 @@ Here are all the solutions mentioned above in table form. IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level -------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -GKE | | | GCE | [docs](https://cloud.google.com/container-engine) | [✓][3] | Commercial -Vagrant | Saltstack | Fedora | flannel | [docs](vagrant.html) | [✓][2] | Project -GCE | Saltstack | Debian | GCE | [docs](gce.html) | [✓][1] | Project -Azure | CoreOS | CoreOS | Weave | [docs](coreos/azure/README.html) | | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) -Docker Single Node | custom | N/A | local | [docs](docker.html) | | Project ([@brendandburns](https://github.com/brendandburns)) -Docker Multi Node | Flannel | N/A | local | [docs](docker-multinode.html) | | Project ([@brendandburns](https://github.com/brendandburns)) -Bare-metal | Ansible | Fedora | flannel | [docs](fedora/fedora_ansible_config.html) | | Project -Digital Ocean | custom | Fedora | Calico | [docs](fedora/fedora-calico.html) | | Community (@djosborne) -Bare-metal | custom | Fedora | _none_ | [docs](fedora/fedora_manual_config.html) | | Project -Bare-metal | custom | Fedora | flannel | [docs](fedora/flannel_multi_node_cluster.html) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) -libvirt | custom | Fedora | flannel | [docs](fedora/flannel_multi_node_cluster.html) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) -KVM | custom | Fedora | flannel | [docs](fedora/flannel_multi_node_cluster.html) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) -Mesos/Docker | custom | Ubuntu | Docker | [docs](mesos-docker.html) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) -Mesos/GCE | | | | [docs](mesos.html) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) -DCOS | Marathon | CoreOS/Alpine | custom | [docs](dcos.html) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) -AWS | CoreOS | CoreOS | flannel | [docs](coreos.html) | | Community -GCE | CoreOS | CoreOS | flannel | [docs](coreos.html) | | Community ([@pires](https://github.com/pires)) -Vagrant | CoreOS | CoreOS | flannel | [docs](coreos.html) | | Community ([@pires](https://github.com/pires), [@AntonioMeireles](https://github.com/AntonioMeireles)) -Bare-metal (Offline) | CoreOS | CoreOS | flannel | [docs](coreos/bare_metal_offline.html) | | Community ([@jeffbean](https://github.com/jeffbean)) -Bare-metal | CoreOS | CoreOS | Calico | [docs](coreos/bare_metal_calico.html) | | Community ([@caseydavenport](https://github.com/caseydavenport)) -CloudStack | Ansible | CoreOS | flannel | [docs](cloudstack.html) | | Community ([@runseb](https://github.com/runseb)) -Vmware | | Debian | OVS | [docs](vsphere.html) | | Community ([@pietern](https://github.com/pietern)) -Bare-metal | custom | CentOS | _none_ | [docs](centos/centos_manual_config.html) | | Community ([@coolsvap](https://github.com/coolsvap)) -AWS | Juju | Ubuntu | flannel | [docs](juju.html) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -OpenStack/HPCloud | Juju | Ubuntu | flannel | [docs](juju.html) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -Joyent | Juju | Ubuntu | flannel | [docs](juju.html) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -AWS | Saltstack | Ubuntu | OVS | [docs](aws.html) | | Community ([@justinsb](https://github.com/justinsb)) -Bare-metal | custom | Ubuntu | Calico | [docs](ubuntu-calico.html) | | Community ([@djosborne](https://github.com/djosborne)) -Bare-metal | custom | Ubuntu | flannel | [docs](ubuntu.html) | | Community ([@resouer](https://github.com/resouer), [@WIZARD-CXY](https://github.com/WIZARD-CXY)) -Local | | | _none_ | [docs](locally.html) | | Community ([@preillyme](https://github.com/preillyme)) -libvirt/KVM | CoreOS | CoreOS | libvirt/KVM | [docs](libvirt-coreos.html) | | Community ([@lhuard1A](https://github.com/lhuard1A)) -oVirt | | | | [docs](ovirt.html) | | Community ([@simon3z](https://github.com/simon3z)) -Rackspace | CoreOS | CoreOS | flannel | [docs](rackspace.html) | | Community ([@doublerr](https://github.com/doublerr)) -any | any | any | any | [docs](scratch.html) | | Community ([@erictune](https://github.com/erictune)) +GKE | | | GCE | [docs](https://cloud.google.com/container-engine) | ['��][3] | Commercial +Vagrant | Saltstack | Fedora | flannel | [docs](vagrant) | ['��][2] | Project +GCE | Saltstack | Debian | GCE | [docs](gce) | ['��][1] | Project +Azure | CoreOS | CoreOS | Weave | [docs](/{{page.version}}/docs/getting-started-guides/coreos/azure/README) | | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) +Docker Single Node | custom | N/A | local | [docs](docker) | | Project ([@brendandburns](https://github.com/brendandburns)) +Docker Multi Node | Flannel | N/A | local | [docs](docker-multinode) | | Project ([@brendandburns](https://github.com/brendandburns)) +Bare-metal | Ansible | Fedora | flannel | [docs](/{{page.version}}/docs/getting-started-guides/fedora/fedora_ansible_config) | | Project +Digital Ocean | custom | Fedora | Calico | [docs](/{{page.version}}/docs/getting-started-guides/fedora/fedora-calico) | | Community (@djosborne) +Bare-metal | custom | Fedora | _none_ | [docs](/{{page.version}}/docs/getting-started-guides/fedora/fedora_manual_config) | | Project +Bare-metal | custom | Fedora | flannel | [docs](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) +libvirt | custom | Fedora | flannel | [docs](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) +KVM | custom | Fedora | flannel | [docs](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) +Mesos/Docker | custom | Ubuntu | Docker | [docs](mesos-docker) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) +Mesos/GCE | | | | [docs](mesos) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) +DCOS | Marathon | CoreOS/Alpine | custom | [docs](dcos) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) +AWS | CoreOS | CoreOS | flannel | [docs](coreos) | | Community +GCE | CoreOS | CoreOS | flannel | [docs](coreos) | | Community ([@pires](https://github.com/pires)) +Vagrant | CoreOS | CoreOS | flannel | [docs](coreos) | | Community ([@pires](https://github.com/pires), [@AntonioMeireles](https://github.com/AntonioMeireles)) +Bare-metal (Offline) | CoreOS | CoreOS | flannel | [docs](/{{page.version}}/docs/getting-started-guides/coreos/bare_metal_offline) | | Community ([@jeffbean](https://github.com/jeffbean)) +Bare-metal | CoreOS | CoreOS | Calico | [docs](/{{page.version}}/docs/getting-started-guides/coreos/bare_metal_calico) | | Community ([@caseydavenport](https://github.com/caseydavenport)) +CloudStack | Ansible | CoreOS | flannel | [docs](cloudstack) | | Community ([@runseb](https://github.com/runseb)) +Vmware | | Debian | OVS | [docs](vsphere) | | Community ([@pietern](https://github.com/pietern)) +Bare-metal | custom | CentOS | _none_ | [docs](/{{page.version}}/docs/getting-started-guides/centos/centos_manual_config) | | Community ([@coolsvap](https://github.com/coolsvap)) +AWS | Juju | Ubuntu | flannel | [docs](juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) +OpenStack/HPCloud | Juju | Ubuntu | flannel | [docs](juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) +Joyent | Juju | Ubuntu | flannel | [docs](juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) +AWS | Saltstack | Ubuntu | OVS | [docs](aws) | | Community ([@justinsb](https://github.com/justinsb)) +Bare-metal | custom | Ubuntu | Calico | [docs](ubuntu-calico) | | Community ([@djosborne](https://github.com/djosborne)) +Bare-metal | custom | Ubuntu | flannel | [docs](ubuntu) | | Community ([@resouer](https://github.com/resouer), [@WIZARD-CXY](https://github.com/WIZARD-CXY)) +Local | | | _none_ | [docs](locally) | | Community ([@preillyme](https://github.com/preillyme)) +libvirt/KVM | CoreOS | CoreOS | libvirt/KVM | [docs](libvirt-coreos) | | Community ([@lhuard1A](https://github.com/lhuard1A)) +oVirt | | | | [docs](ovirt) | | Community ([@simon3z](https://github.com/simon3z)) +Rackspace | CoreOS | CoreOS | flannel | [docs](rackspace) | | Community ([@doublerr](https://github.com/doublerr)) +any | any | any | any | [docs](scratch) | | Community ([@erictune](https://github.com/erictune)) *Note*: The above table is ordered by version test/used in notes followed by support level. @@ -159,7 +156,7 @@ Definition of columns: - **OS** is the base operating system of the nodes. - **Config. Mgmt** is the configuration management system that helps install and maintain Kubernetes software on the nodes. -- **Networking** is what implements the [networking model](../../docs/admin/networking.html). Those with networking type +- **Networking** is what implements the [networking model](/{{page.version}}/docs/admin/networking). Those with networking type _none_ may not support more than one node, or may support multiple VM nodes only in the same physical node. - **Conformance** indicates whether a cluster created with this configuration has passed the project's conformance tests for supporting the API and base features of Kubernetes v1.0.0. @@ -171,7 +168,6 @@ Definition of columns: - **Inactive**: No active maintainer. Not recommended for first-time Kubernetes users, and may be deleted soon. - **Notes** is relevant information such as the version of Kubernetes used. - [1]: https://gist.github.com/erictune/4cabc010906afbcc5061 diff --git a/v1.1/docs/getting-started-guides/aws.md b/v1.1/docs/getting-started-guides/aws.md index 32153b7139..5ee60991f6 100644 --- a/v1.1/docs/getting-started-guides/aws.md +++ b/v1.1/docs/getting-started-guides/aws.md @@ -1,120 +1,109 @@ ---- -title: "Getting started on AWS EC2" -section: guides ---- -## Table of Contents - -{% include pagetoc.html %} - -## Prerequisites - -1. You need an AWS account. Visit [http://aws.amazon.com](http://aws.amazon.com) to get started -2. Install and configure [AWS Command Line Interface](http://aws.amazon.com/cli) -3. You need an AWS [instance profile and role](http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html) with EC2 full access. - -NOTE: This script use the 'default' AWS profile by default. -You may explicitly set AWS profile to use using the `AWS_DEFAULT_PROFILE` environment variable: - -{% highlight bash %} -{% raw %} -export AWS_DEFAULT_PROFILE=myawsprofile -{% endraw %} -{% endhighlight %} - -## Cluster turnup - -### Supported procedure: `get-kube` - -{% highlight bash %} -{% raw %} -#Using wget -export KUBERNETES_PROVIDER=aws; wget -q -O - https://get.k8s.io | bash - -#Using cURL -export KUBERNETES_PROVIDER=aws; curl -sS https://get.k8s.io | bash -{% endraw %} -{% endhighlight %} - -NOTE: This script calls [cluster/kube-up.sh](http://releases.k8s.io/release-1.1/cluster/kube-up.sh) -which in turn calls [cluster/aws/util.sh](http://releases.k8s.io/release-1.1/cluster/aws/util.sh) -using [cluster/aws/config-default.sh](http://releases.k8s.io/release-1.1/cluster/aws/config-default.sh). - -This process takes about 5 to 10 minutes. Once the cluster is up, the IP addresses of your master and node(s) will be printed, -as well as information about the default services running in the cluster (monitoring, logging, dns). User credentials and security -tokens are written in `~/.kube/config`, they will be necessary to use the CLI or the HTTP Basic Auth. - -By default, the script will provision a new VPC and a 4 node k8s cluster in us-west-2a (Oregon) with `t2.micro` instances running on Ubuntu. -You can override the variables defined in [config-default.sh](http://releases.k8s.io/release-1.1/cluster/aws/config-default.sh) to change this behavior as follows: - -{% highlight bash %} -{% raw %} -export KUBE_AWS_ZONE=eu-west-1c -export NUM_MINIONS=2 -export MINION_SIZE=m3.medium -export AWS_S3_REGION=eu-west-1 -export AWS_S3_BUCKET=mycompany-kubernetes-artifacts -export INSTANCE_PREFIX=k8s -... -{% endraw %} -{% endhighlight %} - -It will also try to create or reuse a keypair called "kubernetes", and IAM profiles called "kubernetes-master" and "kubernetes-minion". -If these already exist, make sure you want them to be used here. - -NOTE: If using an existing keypair named "kubernetes" then you must set the `AWS_SSH_KEY` key to point to your private key. - -### Alternatives - -A contributed [example](coreos/coreos_multinode_cluster.html) allows you to setup a Kubernetes cluster based on [CoreOS](http://www.coreos.com), using -EC2 with user data (cloud-config). - -## Getting started with your cluster - -### Command line administration tool: `kubectl` - -The cluster startup script will leave you with a `kubernetes` directory on your workstation. -Alternately, you can download the latest Kubernetes release from [this page](https://github.com/kubernetes/kubernetes/releases). - -Next, add the appropriate binary folder to your `PATH` to access kubectl: - -{% highlight bash %} -{% raw %} -# OS X -export PATH=/platforms/darwin/amd64:$PATH - -# Linux -export PATH=/platforms/linux/amd64:$PATH -{% endraw %} -{% endhighlight %} - -An up-to-date documentation page for this tool is available here: [kubectl manual](../../docs/user-guide/kubectl/kubectl.html) - -By default, `kubectl` will use the `kubeconfig` file generated during the cluster startup for authenticating against the API. -For more information, please read [kubeconfig files](../../docs/user-guide/kubeconfig-file.html) - -### Examples - -See [a simple nginx example](../../docs/user-guide/simple-nginx.html) to try out your new cluster. - -The "Guestbook" application is another popular example to get started with Kubernetes: [guestbook example](../../examples/guestbook/) - -For more complete applications, please look in the [examples directory](../../examples/) - -## Tearing down the cluster - -Make sure the environment variables you used to provision your cluster are still exported, then call the following script inside the -`kubernetes` directory: - -{% highlight bash %} -{% raw %} -cluster/kube-down.sh -{% endraw %} -{% endhighlight %} - -## Further reading - -Please see the [Kubernetes docs](../../docs/) for more details on administering -and using a Kubernetes cluster. - - - +--- +title: "Getting started on AWS EC2" +--- + + +{% include pagetoc.html %} + +## Prerequisites + +1. You need an AWS account. Visit [http://aws.amazon.com](http://aws.amazon.com) to get started +2. Install and configure [AWS Command Line Interface](http://aws.amazon.com/cli) +3. You need an AWS [instance profile and role](http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles) with EC2 full access. + +NOTE: This script use the 'default' AWS profile by default. +You may explicitly set AWS profile to use using the `AWS_DEFAULT_PROFILE` environment variable: + +{% highlight bash %} +export AWS_DEFAULT_PROFILE=myawsprofile +{% endhighlight %} + +## Cluster turnup + +### Supported procedure: `get-kube` + +{% highlight bash %} +#Using wget +export KUBERNETES_PROVIDER=aws; wget -q -O - https://get.k8s.io | bash + +#Using cURL +export KUBERNETES_PROVIDER=aws; curl -sS https://get.k8s.io | bash +{% endhighlight %} + +NOTE: This script calls [cluster/kube-up.sh](http://releases.k8s.io/release-1.1/cluster/kube-up.sh) +which in turn calls [cluster/aws/util.sh](http://releases.k8s.io/release-1.1/cluster/aws/util.sh) +using [cluster/aws/config-default.sh](http://releases.k8s.io/release-1.1/cluster/aws/config-default.sh). + +This process takes about 5 to 10 minutes. Once the cluster is up, the IP addresses of your master and node(s) will be printed, +as well as information about the default services running in the cluster (monitoring, logging, dns). User credentials and security +tokens are written in `~/.kube/config`, they will be necessary to use the CLI or the HTTP Basic Auth. + +By default, the script will provision a new VPC and a 4 node k8s cluster in us-west-2a (Oregon) with `t2.micro` instances running on Ubuntu. +You can override the variables defined in [config-default.sh](http://releases.k8s.io/release-1.1/cluster/aws/config-default.sh) to change this behavior as follows: + +{% highlight bash %} +export KUBE_AWS_ZONE=eu-west-1c +export NUM_MINIONS=2 +export MINION_SIZE=m3.medium +export AWS_S3_REGION=eu-west-1 +export AWS_S3_BUCKET=mycompany-kubernetes-artifacts +export INSTANCE_PREFIX=k8s +... +{% endhighlight %} + +It will also try to create or reuse a keypair called "kubernetes", and IAM profiles called "kubernetes-master" and "kubernetes-minion". +If these already exist, make sure you want them to be used here. + +NOTE: If using an existing keypair named "kubernetes" then you must set the `AWS_SSH_KEY` key to point to your private key. + +### Alternatives + +A contributed [example](/{{page.version}}/docs/getting-started-guides/coreos/coreos_multinode_cluster) allows you to setup a Kubernetes cluster based on [CoreOS](http://www.coreos.com), using +EC2 with user data (cloud-config). + +## Getting started with your cluster + +### Command line administration tool: `kubectl` + +The cluster startup script will leave you with a `kubernetes` directory on your workstation. +Alternately, you can download the latest Kubernetes release from [this page](https://github.com/kubernetes/kubernetes/releases). + +Next, add the appropriate binary folder to your `PATH` to access kubectl: + +{% highlight bash %} +# OS X +export PATH=/platforms/darwin/amd64:$PATH + +# Linux +export PATH=/platforms/linux/amd64:$PATH +{% endhighlight %} + +An up-to-date documentation page for this tool is available here: [kubectl manual](/{{page.version}}/docs/user-guide/kubectl/kubectl) + +By default, `kubectl` will use the `kubeconfig` file generated during the cluster startup for authenticating against the API. +For more information, please read [kubeconfig files](/{{page.version}}/docs/user-guide/kubeconfig-file) + +### Examples + +See [a simple nginx example](/{{page.version}}/docs/user-guide/simple-nginx) to try out your new cluster. + +The "Guestbook" application is another popular example to get started with Kubernetes: [guestbook example](../../examples/guestbook/) + +For more complete applications, please look in the [examples directory](../../examples/) + +## Tearing down the cluster + +Make sure the environment variables you used to provision your cluster are still exported, then call the following script inside the +`kubernetes` directory: + +{% highlight bash %} +cluster/kube-down.sh +{% endhighlight %} + +## Further reading + +Please see the [Kubernetes docs](/{{page.version}}/docs/) for more details on administering +and using a Kubernetes cluster. + + + diff --git a/v1.1/docs/getting-started-guides/azure.md b/v1.1/docs/getting-started-guides/azure.md index a895047e93..b93238d81d 100644 --- a/v1.1/docs/getting-started-guides/azure.md +++ b/v1.1/docs/getting-started-guides/azure.md @@ -6,7 +6,7 @@ title: "Getting started on Microsoft Azure" Getting started on Microsoft Azure ---------------------------------- -Checkout the [coreos azure getting started guide](coreos/azure/README.html) +Checkout the [coreos azure getting started guide](/{{page.version}}/docs/getting-started-guides/coreos/azure/README) diff --git a/v1.1/docs/getting-started-guides/binary_release.md b/v1.1/docs/getting-started-guides/binary_release.md index c50319cefe..7233adfc84 100644 --- a/v1.1/docs/getting-started-guides/binary_release.md +++ b/v1.1/docs/getting-started-guides/binary_release.md @@ -1,10 +1,6 @@ --- title: "Getting a Binary Release" --- - - -## Getting a Binary Release - You can either build a release from sources or download a pre-built release. If you do not plan on developing Kubernetes itself, we suggest a pre-built release. ### Prebuilt Binary Release @@ -20,11 +16,11 @@ Get the Kubernetes source. If you are simply building a release from source the Building a release is simple. {% highlight bash %} -{% raw %} + git clone https://github.com/kubernetes/kubernetes.git cd kubernetes make release -{% endraw %} + {% endhighlight %} For more details on the release process see the [`build/` directory](http://releases.k8s.io/release-1.1/build/) diff --git a/v1.1/docs/getting-started-guides/centos/centos_manual_config.md b/v1.1/docs/getting-started-guides/centos/centos_manual_config.md index b768ff8a25..7d32e246f9 100644 --- a/v1.1/docs/getting-started-guides/centos/centos_manual_config.md +++ b/v1.1/docs/getting-started-guides/centos/centos_manual_config.md @@ -1,199 +1,172 @@ ---- -title: "Getting started on CentOS" -section: guides ---- -## Table of Contents - -{% include pagetoc.html %} - -## Prerequisites - -You need two machines with CentOS installed on them. - -## Starting a cluster - -This is a getting started guide for CentOS. It is a manual configuration so you understand all the underlying packages / services / ports, etc... - -This guide will only get ONE node working. Multiple nodes requires a functional [networking configuration](../../admin/networking.html) done outside of kubernetes. Although the additional Kubernetes configuration requirements should be obvious. - -The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd and the configuration resides in a central location: /etc/kubernetes. We will break the services up between the hosts. The first host, centos-master, will be the Kubernetes master. This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master will also run _etcd_. The remaining host, centos-minion will be the node and run kubelet, proxy, cadvisor and docker. - -**System Information:** - -Hosts: - -``` -{% raw %} -centos-master = 192.168.121.9 -centos-minion = 192.168.121.65 -{% endraw %} -``` - -**Prepare the hosts:** - -* Create virt7-testing repo on all hosts - centos-{master,minion} with following information. - -``` -{% raw %} -[virt7-testing] -name=virt7-testing -baseurl=http://cbs.centos.org/repos/virt7-testing/x86_64/os/ -gpgcheck=0 -{% endraw %} -``` - -* Install Kubernetes on all hosts - centos-{master,minion}. This will also pull in etcd, docker, and cadvisor. - -{% highlight sh %} -{% raw %} -yum -y install --enablerepo=virt7-testing kubernetes -{% endraw %} -{% endhighlight %} - -* Note * Using etcd-0.4.6-7 (This is temporary update in documentation) - -If you do not get etcd-0.4.6-7 installed with virt7-testing repo, - -In the current virt7-testing repo, the etcd package is updated which causes service failure. To avoid this, - -{% highlight sh %} -{% raw %} -yum erase etcd -{% endraw %} -{% endhighlight %} - -It will uninstall the current available etcd package - -{% highlight sh %} -{% raw %} -yum install http://cbs.centos.org/kojifiles/packages/etcd/0.4.6/7.el7.centos/x86_64/etcd-0.4.6-7.el7.centos.x86_64.rpm -yum -y install --enablerepo=virt7-testing kubernetes -{% endraw %} -{% endhighlight %} - -* Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS) - -{% highlight sh %} -{% raw %} -echo "192.168.121.9 centos-master -192.168.121.65 centos-minion" >> /etc/hosts -{% endraw %} -{% endhighlight %} - -* Edit /etc/kubernetes/config which will be the same on all hosts to contain: - -{% highlight sh %} -{% raw %} -# Comma separated list of nodes in the etcd cluster -KUBE_ETCD_SERVERS="--etcd-servers=http://centos-master:4001" - -# logging to stderr means we get it in the systemd journal -KUBE_LOGTOSTDERR="--logtostderr=true" - -# journal message level, 0 is debug -KUBE_LOG_LEVEL="--v=0" - -# Should this cluster be allowed to run privileged docker containers -KUBE_ALLOW_PRIV="--allow-privileged=false" -{% endraw %} -{% endhighlight %} - -* Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers - -{% highlight sh %} -{% raw %} -systemctl disable iptables-services firewalld -systemctl stop iptables-services firewalld -{% endraw %} -{% endhighlight %} - -**Configure the Kubernetes services on the master.** - -* Edit /etc/kubernetes/apiserver to appear as such: - -{% highlight sh %} -{% raw %} -# The address on the local server to listen to. -KUBE_API_ADDRESS="--address=0.0.0.0" - -# The port on the local server to listen on. -KUBE_API_PORT="--port=8080" - -# How the replication controller and scheduler find the kube-apiserver -KUBE_MASTER="--master=http://centos-master:8080" - -# Port kubelets listen on -KUBELET_PORT="--kubelet-port=10250" - -# Address range to use for services -KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" - -# Add your own! -KUBE_API_ARGS="" -{% endraw %} -{% endhighlight %} - -* Start the appropriate services on master: - -{% highlight sh %} -{% raw %} -for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do - systemctl restart $SERVICES - systemctl enable $SERVICES - systemctl status $SERVICES -done -{% endraw %} -{% endhighlight %} - -**Configure the Kubernetes services on the node.** - -***We need to configure the kubelet and start the kubelet and proxy*** - -* Edit /etc/kubernetes/kubelet to appear as such: - -{% highlight sh %} -{% raw %} -# The address for the info server to serve on -KUBELET_ADDRESS="--address=0.0.0.0" - -# The port for the info server to serve on -KUBELET_PORT="--port=10250" - -# You may leave this blank to use the actual hostname -KUBELET_HOSTNAME="--hostname-override=centos-minion" - -# Location of the api-server -KUBELET_API_SERVER="--api-servers=http://centos-master:8080" - -# Add your own! -KUBELET_ARGS="" -{% endraw %} -{% endhighlight %} - -* Start the appropriate services on node (centos-minion). - -{% highlight sh %} -{% raw %} -for SERVICES in kube-proxy kubelet docker; do - systemctl restart $SERVICES - systemctl enable $SERVICES - systemctl status $SERVICES -done -{% endraw %} -{% endhighlight %} - -*You should be finished!* - -* Check to make sure the cluster can see the node (on centos-master) - -{% highlight console %} -{% raw %} -$ kubectl get nodes -NAME LABELS STATUS -centos-minion Ready -{% endraw %} -{% endhighlight %} - -**The cluster should be running! Launch a test pod.** - -You should have a functional cluster, check out [101](../../../docs/user-guide/walkthrough/README.html)! \ No newline at end of file +--- +title: "Getting started on CentOS" +--- + + +{% include pagetoc.html %} + +## Prerequisites + +You need two machines with CentOS installed on them. + +## Starting a cluster + +This is a getting started guide for CentOS. It is a manual configuration so you understand all the underlying packages / services / ports, etc... + +This guide will only get ONE node working. Multiple nodes requires a functional [networking configuration](../../admin/networking) done outside of kubernetes. Although the additional Kubernetes configuration requirements should be obvious. + +The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd and the configuration resides in a central location: /etc/kubernetes. We will break the services up between the hosts. The first host, centos-master, will be the Kubernetes master. This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master will also run _etcd_. The remaining host, centos-minion will be the node and run kubelet, proxy, cadvisor and docker. + +**System Information:** + +Hosts: + +``` +centos-master = 192.168.121.9 +centos-minion = 192.168.121.65 +``` + +**Prepare the hosts:** + +* Create virt7-testing repo on all hosts - centos-{master,minion} with following information. + +``` +[virt7-testing] +name=virt7-testing +baseurl=http://cbs.centos.org/repos/virt7-testing/x86_64/os/ +gpgcheck=0 +``` + +* Install Kubernetes on all hosts - centos-{master,minion}. This will also pull in etcd, docker, and cadvisor. + +{% highlight sh %} +yum -y install --enablerepo=virt7-testing kubernetes +{% endhighlight %} + +* Note * Using etcd-0.4.6-7 (This is temporary update in documentation) + +If you do not get etcd-0.4.6-7 installed with virt7-testing repo, + +In the current virt7-testing repo, the etcd package is updated which causes service failure. To avoid this, + +{% highlight sh %} +yum erase etcd +{% endhighlight %} + +It will uninstall the current available etcd package + +{% highlight sh %} +yum install http://cbs.centos.org/kojifiles/packages/etcd/0.4.6/7.el7.centos/x86_64/etcd-0.4.6-7.el7.centos.x86_64.rpm +yum -y install --enablerepo=virt7-testing kubernetes +{% endhighlight %} + +* Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS) + +{% highlight sh %} +echo "192.168.121.9 centos-master +192.168.121.65 centos-minion" >> /etc/hosts +{% endhighlight %} + +* Edit /etc/kubernetes/config which will be the same on all hosts to contain: + +{% highlight sh %} +# Comma separated list of nodes in the etcd cluster +KUBE_ETCD_SERVERS="--etcd-servers=http://centos-master:4001" + +# logging to stderr means we get it in the systemd journal +KUBE_LOGTOSTDERR="--logtostderr=true" + +# journal message level, 0 is debug +KUBE_LOG_LEVEL="--v=0" + +# Should this cluster be allowed to run privileged docker containers +KUBE_ALLOW_PRIV="--allow-privileged=false" +{% endhighlight %} + +* Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers + +{% highlight sh %} +systemctl disable iptables-services firewalld +systemctl stop iptables-services firewalld +{% endhighlight %} + +**Configure the Kubernetes services on the master.** + +* Edit /etc/kubernetes/apiserver to appear as such: + +{% highlight sh %} +# The address on the local server to listen to. +KUBE_API_ADDRESS="--address=0.0.0.0" + +# The port on the local server to listen on. +KUBE_API_PORT="--port=8080" + +# How the replication controller and scheduler find the kube-apiserver +KUBE_MASTER="--master=http://centos-master:8080" + +# Port kubelets listen on +KUBELET_PORT="--kubelet-port=10250" + +# Address range to use for services +KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" + +# Add your own! +KUBE_API_ARGS="" +{% endhighlight %} + +* Start the appropriate services on master: + +{% highlight sh %} +for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do + systemctl restart $SERVICES + systemctl enable $SERVICES + systemctl status $SERVICES +done +{% endhighlight %} + +**Configure the Kubernetes services on the node.** + +***We need to configure the kubelet and start the kubelet and proxy*** + +* Edit /etc/kubernetes/kubelet to appear as such: + +{% highlight sh %} +# The address for the info server to serve on +KUBELET_ADDRESS="--address=0.0.0.0" + +# The port for the info server to serve on +KUBELET_PORT="--port=10250" + +# You may leave this blank to use the actual hostname +KUBELET_HOSTNAME="--hostname-override=centos-minion" + +# Location of the api-server +KUBELET_API_SERVER="--api-servers=http://centos-master:8080" + +# Add your own! +KUBELET_ARGS="" +{% endhighlight %} + +* Start the appropriate services on node (centos-minion). + +{% highlight sh %} +for SERVICES in kube-proxy kubelet docker; do + systemctl restart $SERVICES + systemctl enable $SERVICES + systemctl status $SERVICES +done +{% endhighlight %} + +*You should be finished!* + +* Check to make sure the cluster can see the node (on centos-master) + +{% highlight console %} +$ kubectl get nodes +NAME LABELS STATUS +centos-minion Ready +{% endhighlight %} + +**The cluster should be running! Launch a test pod.** + +You should have a functional cluster, check out [101](/{{page.version}}/docs/user-guide/walkthrough/README)! \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/cloudstack.md b/v1.1/docs/getting-started-guides/cloudstack.md index 3ca85c7c39..416bf46ea1 100644 --- a/v1.1/docs/getting-started-guides/cloudstack.md +++ b/v1.1/docs/getting-started-guides/cloudstack.md @@ -1,21 +1,20 @@ --- title: "Getting started on CloudStack" -section: guides --- ## Introduction CloudStack is a software to build public and private clouds based on hardware virtualization principles (traditional IaaS). To deploy Kubernetes on CloudStack there are several possibilities depending on the Cloud being used and what images are made available. [Exoscale](http://exoscale.ch) for instance makes a [CoreOS](http://coreos.com) template available, therefore instructions to deploy Kubernetes on coreOS can be used. CloudStack also has a vagrant plugin available, hence Vagrant could be used to deploy Kubernetes either using the existing shell provisioner or using new Salt based recipes. -[CoreOS](http://coreos.com) templates for CloudStack are built [nightly](http://stable.release.core-os.net/amd64-usr/current/). CloudStack operators need to [register](http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/latest/templates.html) this template in their cloud before proceeding with these Kubernetes deployment instructions. +[CoreOS](http://coreos.com) templates for CloudStack are built [nightly](http://stable.release.core-os.net/amd64-usr/current/). CloudStack operators need to [register](http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/latest/templates) this template in their cloud before proceeding with these Kubernetes deployment instructions. This guide uses an [Ansible playbook](https://github.com/runseb/ansible-kubernetes). -This is a completely automated, a single playbook deploys Kubernetes based on the coreOS [instructions](coreos/coreos_multinode_cluster.html). +This is a completely automated, a single playbook deploys Kubernetes based on the coreOS [instructions](/{{page.version}}/docs/getting-started-guides/coreos/coreos_multinode_cluster). This [Ansible](http://ansibleworks.com) playbook deploys Kubernetes on a CloudStack based Cloud using CoreOS images. The playbook, creates an ssh key pair, creates a security group and associated rules and finally starts coreOS instances configured via cloud-init. -## Table of Contents + {% include pagetoc.html %} diff --git a/v1.1/docs/getting-started-guides/coreos.md b/v1.1/docs/getting-started-guides/coreos.md index f2718b80b3..070690eea4 100644 --- a/v1.1/docs/getting-started-guides/coreos.md +++ b/v1.1/docs/getting-started-guides/coreos.md @@ -1,9 +1,8 @@ --- title: "Getting Started on CoreOS" -section: guides --- -## Table of Contents + {% include pagetoc.html %} @@ -11,21 +10,21 @@ There are multiple guides on running Kubernetes with [CoreOS](https://coreos.com ### Official CoreOS Guides -These guides are maintained by CoreOS and deploy Kubernetes the "CoreOS Way" with full TLS, the DNS add-on, and more. These guides pass Kubernetes conformance testing and we encourage you to [test this yourself](https://coreos.com/kubernetes/docs/latest/conformance-tests.html). +These guides are maintained by CoreOS and deploy Kubernetes the "CoreOS Way" with full TLS, the DNS add-on, and more. These guides pass Kubernetes conformance testing and we encourage you to [test this yourself](https://coreos.com/kubernetes/docs/latest/conformance-tests). -[**Vagrant Multi-Node**](https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant.html) +[**Vagrant Multi-Node**](https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant) Guide to setting up a multi-node cluster on Vagrant. The deployer can independently configure the number of etcd nodes, master nodes, and worker nodes to bring up a fully HA control plane.
    -[**Vagrant Single-Node**](https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant-single.html) +[**Vagrant Single-Node**](https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant-single) The quickest way to set up a Kubernetes development environment locally. As easy as `git clone`, `vagrant up` and configuring `kubectl`.
    -[**Full Step by Step Guide**](https://coreos.com/kubernetes/docs/latest/getting-started.html) +[**Full Step by Step Guide**](https://coreos.com/kubernetes/docs/latest/getting-started) A generic guide to setting up an HA cluster on any cloud or bare metal, with full TLS. Repeat the master or worker steps to configure more machines of that role. @@ -33,7 +32,7 @@ A generic guide to setting up an HA cluster on any cloud or bare metal, with ful These guides are maintained by community members, cover specific platforms and use cases, and experiment with different ways of configuring Kubernetes on CoreOS. -[**Multi-node Cluster**](coreos/coreos_multinode_cluster.html) +[**Multi-node Cluster**](/{{page.version}}/docs/getting-started-guides/coreos/coreos_multinode_cluster) Set up a single master, multi-worker cluster on your choice of platform: AWS, GCE, or VMware Fusion. @@ -63,7 +62,7 @@ Guide to running a single master, multi-worker cluster controlled by an OS X men
    -[**Resizable multi-node cluster on Azure with Weave**](coreos/azure/README.html) +[**Resizable multi-node cluster on Azure with Weave**](/{{page.version}}/docs/getting-started-guides/coreos/azure/README) Guide to running an HA etcd cluster with a single master on Azure. Uses the Azure node.js CLI to resize the cluster. diff --git a/v1.1/docs/getting-started-guides/coreos/azure/README.md b/v1.1/docs/getting-started-guides/coreos/azure/README.md index adb215443d..e168f7e583 100644 --- a/v1.1/docs/getting-started-guides/coreos/azure/README.md +++ b/v1.1/docs/getting-started-guides/coreos/azure/README.md @@ -1,263 +1,226 @@ ---- -title: "Kubernetes on Azure with CoreOS and Weave" -section: guides ---- - -## Introduction - -In this guide I will demonstrate how to deploy a Kubernetes cluster to Azure cloud. You will be using CoreOS with Weave, which implements simple and secure networking, in a transparent, yet robust way. The purpose of this guide is to provide an out-of-the-box implementation that can ultimately be taken into production with little change. It will demonstrate how to provision a dedicated Kubernetes master and etcd nodes, and show how to scale the cluster with ease. - -## Table of Contents - -{% include pagetoc.html %} - -### Prerequisites - -1. You need an Azure account. - -## Let's go! - -To get started, you need to checkout the code: - -{% highlight sh %} -{% raw %} -git clone https://github.com/kubernetes/kubernetes -cd kubernetes/docs/getting-started-guides/coreos/azure/ -{% endraw %} -{% endhighlight %} - -You will need to have [Node.js installed](http://nodejs.org/download/) on you machine. If you have previously used Azure CLI, you should have it already. - -First, you need to install some of the dependencies with - -{% highlight sh %} -{% raw %} -npm install -{% endraw %} -{% endhighlight %} - -Now, all you need to do is: - -{% highlight sh %} -{% raw %} -./azure-login.js -u -./create-kubernetes-cluster.js -{% endraw %} -{% endhighlight %} - -This script will provision a cluster suitable for production use, where there is a ring of 3 dedicated etcd nodes: 1 kubernetes master and 2 kubernetes nodes. The `kube-00` VM will be the master, your work loads are only to be deployed on the nodes, `kube-01` and `kube-02`. Initially, all VMs are single-core, to ensure a user of the free tier can reproduce it without paying extra. I will show how to add more bigger VMs later. - -![VMs in Azure](initial_cluster.png) - -Once the creation of Azure VMs has finished, you should see the following: - -{% highlight console %} -{% raw %} -... -azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_1c1496016083b4_ssh_conf ` -azure_wrapper/info: The hosts in this deployment are: - [ 'etcd-00', 'etcd-01', 'etcd-02', 'kube-00', 'kube-01', 'kube-02' ] -azure_wrapper/info: Saved state into `./output/kube_1c1496016083b4_deployment.yml` -{% endraw %} -{% endhighlight %} - -Let's login to the master node like so: - -{% highlight sh %} -{% raw %} -ssh -F ./output/kube_1c1496016083b4_ssh_conf kube-00 -{% endraw %} -{% endhighlight %} - -> Note: config file name will be different, make sure to use the one you see. - -Check there are 2 nodes in the cluster: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ kubectl get nodes -NAME LABELS STATUS -kube-01 kubernetes.io/hostname=kube-01 Ready -kube-02 kubernetes.io/hostname=kube-02 Ready -{% endraw %} -{% endhighlight %} - -## Deploying the workload - -Let's follow the Guestbook example now: - -{% highlight sh %} -{% raw %} -kubectl create -f ~/guestbook-example -{% endraw %} -{% endhighlight %} - -You need to wait for the pods to get deployed, run the following and wait for `STATUS` to change from `Pending` to `Running`. - -{% highlight sh %} -{% raw %} -kubectl get pods --watch -{% endraw %} -{% endhighlight %} - -> Note: the most time it will spend downloading Docker container images on each of the nodes. - -Eventually you should see: - -{% highlight console %} -{% raw %} -NAME READY STATUS RESTARTS AGE -frontend-0a9xi 1/1 Running 0 4m -frontend-4wahe 1/1 Running 0 4m -frontend-6l36j 1/1 Running 0 4m -redis-master-talmr 1/1 Running 0 4m -redis-slave-12zfd 1/1 Running 0 4m -redis-slave-3nbce 1/1 Running 0 4m -{% endraw %} -{% endhighlight %} - -## Scaling - -Two single-core nodes are certainly not enough for a production system of today. Let's scale the cluster by adding a couple of bigger nodes. - -You will need to open another terminal window on your machine and go to the same working directory (e.g. `~/Workspace/kubernetes/docs/getting-started-guides/coreos/azure/`). - -First, lets set the size of new VMs: - -{% highlight sh %} -{% raw %} -export AZ_VM_SIZE=Large -{% endraw %} -{% endhighlight %} - -Now, run scale script with state file of the previous deployment and number of nodes to add: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ ./scale-kubernetes-cluster.js ./output/kube_1c1496016083b4_deployment.yml 2 -... -azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_8f984af944f572_ssh_conf ` -azure_wrapper/info: The hosts in this deployment are: - [ 'etcd-00', - 'etcd-01', - 'etcd-02', - 'kube-00', - 'kube-01', - 'kube-02', - 'kube-03', - 'kube-04' ] -azure_wrapper/info: Saved state into `./output/kube_8f984af944f572_deployment.yml` -{% endraw %} -{% endhighlight %} - -> Note: this step has created new files in `./output`. - -Back on `kube-00`: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ kubectl get nodes -NAME LABELS STATUS -kube-01 kubernetes.io/hostname=kube-01 Ready -kube-02 kubernetes.io/hostname=kube-02 Ready -kube-03 kubernetes.io/hostname=kube-03 Ready -kube-04 kubernetes.io/hostname=kube-04 Ready -{% endraw %} -{% endhighlight %} - -You can see that two more nodes joined happily. Let's scale the number of Guestbook instances now. - -First, double-check how many replication controllers there are: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ kubectl get rc -ONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 3 -redis-master master redis name=redis-master 1 -redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 2 -{% endraw %} -{% endhighlight %} - -As there are 4 nodes, let's scale proportionally: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ kubectl scale --replicas=4 rc redis-slave ->>>>>>> coreos/azure: Updates for 1.0 -scaled -core@kube-00 ~ $ kubectl scale --replicas=4 rc frontend -scaled -{% endraw %} -{% endhighlight %} - -Check what you have now: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ kubectl get rc -CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 4 -redis-master master redis name=redis-master 1 -redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 4 -{% endraw %} -{% endhighlight %} - -You now will have more instances of front-end Guestbook apps and Redis slaves; and, if you look up all pods labeled `name=frontend`, you should see one running on each node. - -{% highlight console %} -{% raw %} -core@kube-00 ~/guestbook-example $ kubectl get pods -l name=frontend -NAME READY STATUS RESTARTS AGE -frontend-0a9xi 1/1 Running 0 22m -frontend-4wahe 1/1 Running 0 22m -frontend-6l36j 1/1 Running 0 22m -frontend-z9oxo 1/1 Running 0 41s -{% endraw %} -{% endhighlight %} - -## Exposing the app to the outside world - -There is no native Azure load-balancer support in Kubernetes 1.0, however here is how you can expose the Guestbook app to the Internet. - -``` -{% raw %} -./expose_guestbook_app_port.sh ./output/kube_1c1496016083b4_ssh_conf -Guestbook app is on port 31605, will map it to port 80 on kube-00 -info: Executing command vm endpoint create -+ Getting virtual machines -+ Reading network configuration -+ Updating network configuration -info: vm endpoint create command OK -info: Executing command vm endpoint show -+ Getting virtual machines -data: Name : tcp-80-31605 -data: Local port : 31605 -data: Protcol : tcp -data: Virtual IP Address : 137.117.156.164 -data: Direct server return : Disabled -info: vm endpoint show command OK -{% endraw %} -``` - -You then should be able to access it from anywhere via the Azure virtual IP for `kube-00` displayed above, i.e. `http://137.117.156.164/` in my case. - -## Next steps - -You now have a full-blow cluster running in Azure, congrats! - -You should probably try deploy other [example apps](../../../../examples/) or write your own ;) - -## Tear down... - -If you don't wish care about the Azure bill, you can tear down the cluster. It's easy to redeploy it, as you can see. - -{% highlight sh %} -{% raw %} -./destroy-cluster.js ./output/kube_8f984af944f572_deployment.yml -{% endraw %} -{% endhighlight %} - -> Note: make sure to use the _latest state file_, as after scaling there is a new one. - +--- +title: "Kubernetes on Azure with CoreOS and Weave" +--- + +## Introduction + +In this guide I will demonstrate how to deploy a Kubernetes cluster to Azure cloud. You will be using CoreOS with Weave, which implements simple and secure networking, in a transparent, yet robust way. The purpose of this guide is to provide an out-of-the-box implementation that can ultimately be taken into production with little change. It will demonstrate how to provision a dedicated Kubernetes master and etcd nodes, and show how to scale the cluster with ease. + + + +{% include pagetoc.html %} + +### Prerequisites + +1. You need an Azure account. + +## Let's go! + +To get started, you need to checkout the code: + +{% highlight sh %} +git clone https://github.com/kubernetes/kubernetes +cd kubernetes/docs/getting-started-guides/coreos/azure/ +{% endhighlight %} + +You will need to have [Node.js installed](http://nodejs.org/download/) on you machine. If you have previously used Azure CLI, you should have it already. + +First, you need to install some of the dependencies with + +{% highlight sh %} +npm install +{% endhighlight %} + +Now, all you need to do is: + +{% highlight sh %} +./azure-login.js -u +./create-kubernetes-cluster.js +{% endhighlight %} + +This script will provision a cluster suitable for production use, where there is a ring of 3 dedicated etcd nodes: 1 kubernetes master and 2 kubernetes nodes. The `kube-00` VM will be the master, your work loads are only to be deployed on the nodes, `kube-01` and `kube-02`. Initially, all VMs are single-core, to ensure a user of the free tier can reproduce it without paying extra. I will show how to add more bigger VMs later. + +![VMs in Azure](initial_cluster.png) + +Once the creation of Azure VMs has finished, you should see the following: + +{% highlight console %} +... +azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_1c1496016083b4_ssh_conf ` +azure_wrapper/info: The hosts in this deployment are: + [ 'etcd-00', 'etcd-01', 'etcd-02', 'kube-00', 'kube-01', 'kube-02' ] +azure_wrapper/info: Saved state into `./output/kube_1c1496016083b4_deployment.yml` +{% endhighlight %} + +Let's login to the master node like so: + +{% highlight sh %} +ssh -F ./output/kube_1c1496016083b4_ssh_conf kube-00 +{% endhighlight %} + +> Note: config file name will be different, make sure to use the one you see. + +Check there are 2 nodes in the cluster: + +{% highlight console %} +core@kube-00 ~ $ kubectl get nodes +NAME LABELS STATUS +kube-01 kubernetes.io/hostname=kube-01 Ready +kube-02 kubernetes.io/hostname=kube-02 Ready +{% endhighlight %} + +## Deploying the workload + +Let's follow the Guestbook example now: + +{% highlight sh %} +kubectl create -f ~/guestbook-example +{% endhighlight %} + +You need to wait for the pods to get deployed, run the following and wait for `STATUS` to change from `Pending` to `Running`. + +{% highlight sh %} +kubectl get pods --watch +{% endhighlight %} + +> Note: the most time it will spend downloading Docker container images on each of the nodes. + +Eventually you should see: + +{% highlight console %} +NAME READY STATUS RESTARTS AGE +frontend-0a9xi 1/1 Running 0 4m +frontend-4wahe 1/1 Running 0 4m +frontend-6l36j 1/1 Running 0 4m +redis-master-talmr 1/1 Running 0 4m +redis-slave-12zfd 1/1 Running 0 4m +redis-slave-3nbce 1/1 Running 0 4m +{% endhighlight %} + +## Scaling + +Two single-core nodes are certainly not enough for a production system of today. Let's scale the cluster by adding a couple of bigger nodes. + +You will need to open another terminal window on your machine and go to the same working directory (e.g. `~/Workspace/kubernetes/docs/getting-started-guides/coreos/azure/`). + +First, lets set the size of new VMs: + +{% highlight sh %} +export AZ_VM_SIZE=Large +{% endhighlight %} + +Now, run scale script with state file of the previous deployment and number of nodes to add: + +{% highlight console %} +core@kube-00 ~ $ ./scale-kubernetes-cluster.js ./output/kube_1c1496016083b4_deployment.yml 2 +... +azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_8f984af944f572_ssh_conf ` +azure_wrapper/info: The hosts in this deployment are: + [ 'etcd-00', + 'etcd-01', + 'etcd-02', + 'kube-00', + 'kube-01', + 'kube-02', + 'kube-03', + 'kube-04' ] +azure_wrapper/info: Saved state into `./output/kube_8f984af944f572_deployment.yml` +{% endhighlight %} + +> Note: this step has created new files in `./output`. + +Back on `kube-00`: + +{% highlight console %} +core@kube-00 ~ $ kubectl get nodes +NAME LABELS STATUS +kube-01 kubernetes.io/hostname=kube-01 Ready +kube-02 kubernetes.io/hostname=kube-02 Ready +kube-03 kubernetes.io/hostname=kube-03 Ready +kube-04 kubernetes.io/hostname=kube-04 Ready +{% endhighlight %} + +You can see that two more nodes joined happily. Let's scale the number of Guestbook instances now. + +First, double-check how many replication controllers there are: + +{% highlight console %} +core@kube-00 ~ $ kubectl get rc +ONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS +frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 3 +redis-master master redis name=redis-master 1 +redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 2 +{% endhighlight %} + +As there are 4 nodes, let's scale proportionally: + +{% highlight console %} +core@kube-00 ~ $ kubectl scale --replicas=4 rc redis-slave +>>>>>>> coreos/azure: Updates for 1.0 +scaled +core@kube-00 ~ $ kubectl scale --replicas=4 rc frontend +scaled +{% endhighlight %} + +Check what you have now: + +{% highlight console %} +core@kube-00 ~ $ kubectl get rc +CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS +frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 4 +redis-master master redis name=redis-master 1 +redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 4 +{% endhighlight %} + +You now will have more instances of front-end Guestbook apps and Redis slaves; and, if you look up all pods labeled `name=frontend`, you should see one running on each node. + +{% highlight console %} +core@kube-00 ~/guestbook-example $ kubectl get pods -l name=frontend +NAME READY STATUS RESTARTS AGE +frontend-0a9xi 1/1 Running 0 22m +frontend-4wahe 1/1 Running 0 22m +frontend-6l36j 1/1 Running 0 22m +frontend-z9oxo 1/1 Running 0 41s +{% endhighlight %} + +## Exposing the app to the outside world + +There is no native Azure load-balancer support in Kubernetes 1.0, however here is how you can expose the Guestbook app to the Internet. + +``` +./expose_guestbook_app_port.sh ./output/kube_1c1496016083b4_ssh_conf +Guestbook app is on port 31605, will map it to port 80 on kube-00 +info: Executing command vm endpoint create ++ Getting virtual machines ++ Reading network configuration ++ Updating network configuration +info: vm endpoint create command OK +info: Executing command vm endpoint show ++ Getting virtual machines +data: Name : tcp-80-31605 +data: Local port : 31605 +data: Protcol : tcp +data: Virtual IP Address : 137.117.156.164 +data: Direct server return : Disabled +info: vm endpoint show command OK +``` + +You then should be able to access it from anywhere via the Azure virtual IP for `kube-00` displayed above, i.e. `http://137.117.156.164/` in my case. + +## Next steps + +You now have a full-blow cluster running in Azure, congrats! + +You should probably try deploy other [example apps](../../../../examples/) or write your own ;) + +## Tear down... + +If you don't wish care about the Azure bill, you can tear down the cluster. It's easy to redeploy it, as you can see. + +{% highlight sh %} +./destroy-cluster.js ./output/kube_8f984af944f572_deployment.yml +{% endhighlight %} + +> Note: make sure to use the _latest state file_, as after scaling there is a new one. + By the way, with the scripts shown, you can deploy multiple clusters, if you like :) \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/coreos/azure/index.md b/v1.1/docs/getting-started-guides/coreos/azure/index.md index 6329dafaaf..b8371bfc6e 100644 --- a/v1.1/docs/getting-started-guides/coreos/azure/index.md +++ b/v1.1/docs/getting-started-guides/coreos/azure/index.md @@ -1,275 +1,239 @@ ---- -title: "Kubernetes on Azure with CoreOS and Weave" ---- - -Kubernetes on Azure with CoreOS and [Weave](http://weave.works) ---------------------------------------------------------------- - -**Table of Contents** - -- [Introduction](#introduction) -- [Prerequisites](#prerequisites) -- [Let's go!](#lets-go) -- [Deploying the workload](#deploying-the-workload) -- [Scaling](#scaling) -- [Exposing the app to the outside world](#exposing-the-app-to-the-outside-world) -- [Next steps](#next-steps) -- [Tear down...](#tear-down) - -## Introduction - -In this guide I will demonstrate how to deploy a Kubernetes cluster to Azure cloud. You will be using CoreOS with Weave, which implements simple and secure networking, in a transparent, yet robust way. The purpose of this guide is to provide an out-of-the-box implementation that can ultimately be taken into production with little change. It will demonstrate how to provision a dedicated Kubernetes master and etcd nodes, and show how to scale the cluster with ease. - -### Prerequisites - -1. You need an Azure account. - -## Let's go! - -To get started, you need to checkout the code: - -{% highlight sh %} -{% raw %} -git clone https://github.com/kubernetes/kubernetes -cd kubernetes/docs/getting-started-guides/coreos/azure/ -{% endraw %} -{% endhighlight %} - -You will need to have [Node.js installed](http://nodejs.org/download/) on you machine. If you have previously used Azure CLI, you should have it already. - -First, you need to install some of the dependencies with - -{% highlight sh %} -{% raw %} -npm install -{% endraw %} -{% endhighlight %} - -Now, all you need to do is: - -{% highlight sh %} -{% raw %} -./azure-login.js -u -./create-kubernetes-cluster.js -{% endraw %} -{% endhighlight %} - -This script will provision a cluster suitable for production use, where there is a ring of 3 dedicated etcd nodes: 1 kubernetes master and 2 kubernetes nodes. The `kube-00` VM will be the master, your work loads are only to be deployed on the nodes, `kube-01` and `kube-02`. Initially, all VMs are single-core, to ensure a user of the free tier can reproduce it without paying extra. I will show how to add more bigger VMs later. - -![VMs in Azure](initial_cluster.png) - -Once the creation of Azure VMs has finished, you should see the following: - -{% highlight console %} -{% raw %} -... -azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_1c1496016083b4_ssh_conf ` -azure_wrapper/info: The hosts in this deployment are: - [ 'etcd-00', 'etcd-01', 'etcd-02', 'kube-00', 'kube-01', 'kube-02' ] -azure_wrapper/info: Saved state into `./output/kube_1c1496016083b4_deployment.yml` -{% endraw %} -{% endhighlight %} - -Let's login to the master node like so: - -{% highlight sh %} -{% raw %} -ssh -F ./output/kube_1c1496016083b4_ssh_conf kube-00 -{% endraw %} -{% endhighlight %} - -> Note: config file name will be different, make sure to use the one you see. - -Check there are 2 nodes in the cluster: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ kubectl get nodes -NAME LABELS STATUS -kube-01 kubernetes.io/hostname=kube-01 Ready -kube-02 kubernetes.io/hostname=kube-02 Ready -{% endraw %} -{% endhighlight %} - -## Deploying the workload - -Let's follow the Guestbook example now: - -{% highlight sh %} -{% raw %} -kubectl create -f ~/guestbook-example -{% endraw %} -{% endhighlight %} - -You need to wait for the pods to get deployed, run the following and wait for `STATUS` to change from `Pending` to `Running`. - -{% highlight sh %} -{% raw %} -kubectl get pods --watch -{% endraw %} -{% endhighlight %} - -> Note: the most time it will spend downloading Docker container images on each of the nodes. - -Eventually you should see: - -{% highlight console %} -{% raw %} -NAME READY STATUS RESTARTS AGE -frontend-0a9xi 1/1 Running 0 4m -frontend-4wahe 1/1 Running 0 4m -frontend-6l36j 1/1 Running 0 4m -redis-master-talmr 1/1 Running 0 4m -redis-slave-12zfd 1/1 Running 0 4m -redis-slave-3nbce 1/1 Running 0 4m -{% endraw %} -{% endhighlight %} - -## Scaling - -Two single-core nodes are certainly not enough for a production system of today. Let's scale the cluster by adding a couple of bigger nodes. - -You will need to open another terminal window on your machine and go to the same working directory (e.g. `~/Workspace/kubernetes/docs/getting-started-guides/coreos/azure/`). - -First, lets set the size of new VMs: - -{% highlight sh %} -{% raw %} -export AZ_VM_SIZE=Large -{% endraw %} -{% endhighlight %} - -Now, run scale script with state file of the previous deployment and number of nodes to add: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ ./scale-kubernetes-cluster.js ./output/kube_1c1496016083b4_deployment.yml 2 -... -azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_8f984af944f572_ssh_conf ` -azure_wrapper/info: The hosts in this deployment are: - [ 'etcd-00', - 'etcd-01', - 'etcd-02', - 'kube-00', - 'kube-01', - 'kube-02', - 'kube-03', - 'kube-04' ] -azure_wrapper/info: Saved state into `./output/kube_8f984af944f572_deployment.yml` -{% endraw %} -{% endhighlight %} - -> Note: this step has created new files in `./output`. - -Back on `kube-00`: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ kubectl get nodes -NAME LABELS STATUS -kube-01 kubernetes.io/hostname=kube-01 Ready -kube-02 kubernetes.io/hostname=kube-02 Ready -kube-03 kubernetes.io/hostname=kube-03 Ready -kube-04 kubernetes.io/hostname=kube-04 Ready -{% endraw %} -{% endhighlight %} - -You can see that two more nodes joined happily. Let's scale the number of Guestbook instances now. - -First, double-check how many replication controllers there are: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ kubectl get rc -ONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 3 -redis-master master redis name=redis-master 1 -redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 2 -{% endraw %} -{% endhighlight %} - -As there are 4 nodes, let's scale proportionally: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ kubectl scale --replicas=4 rc redis-slave ->>>>>>> coreos/azure: Updates for 1.0 -scaled -core@kube-00 ~ $ kubectl scale --replicas=4 rc frontend -scaled -{% endraw %} -{% endhighlight %} - -Check what you have now: - -{% highlight console %} -{% raw %} -core@kube-00 ~ $ kubectl get rc -CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 4 -redis-master master redis name=redis-master 1 -redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 4 -{% endraw %} -{% endhighlight %} - -You now will have more instances of front-end Guestbook apps and Redis slaves; and, if you look up all pods labeled `name=frontend`, you should see one running on each node. - -{% highlight console %} -{% raw %} -core@kube-00 ~/guestbook-example $ kubectl get pods -l name=frontend -NAME READY STATUS RESTARTS AGE -frontend-0a9xi 1/1 Running 0 22m -frontend-4wahe 1/1 Running 0 22m -frontend-6l36j 1/1 Running 0 22m -frontend-z9oxo 1/1 Running 0 41s -{% endraw %} -{% endhighlight %} - -## Exposing the app to the outside world - -There is no native Azure load-balancer support in Kubernetes 1.0, however here is how you can expose the Guestbook app to the Internet. - -``` -{% raw %} -./expose_guestbook_app_port.sh ./output/kube_1c1496016083b4_ssh_conf -Guestbook app is on port 31605, will map it to port 80 on kube-00 -info: Executing command vm endpoint create -+ Getting virtual machines -+ Reading network configuration -+ Updating network configuration -info: vm endpoint create command OK -info: Executing command vm endpoint show -+ Getting virtual machines -data: Name : tcp-80-31605 -data: Local port : 31605 -data: Protcol : tcp -data: Virtual IP Address : 137.117.156.164 -data: Direct server return : Disabled -info: vm endpoint show command OK -{% endraw %} -``` - -You then should be able to access it from anywhere via the Azure virtual IP for `kube-00` displayed above, i.e. `http://137.117.156.164/` in my case. - -## Next steps - -You now have a full-blow cluster running in Azure, congrats! - -You should probably try deploy other [example apps](../../../../examples/) or write your own ;) - -## Tear down... - -If you don't wish care about the Azure bill, you can tear down the cluster. It's easy to redeploy it, as you can see. - -{% highlight sh %} -{% raw %} -./destroy-cluster.js ./output/kube_8f984af944f572_deployment.yml -{% endraw %} -{% endhighlight %} - -> Note: make sure to use the _latest state file_, as after scaling there is a new one. - -By the way, with the scripts shown, you can deploy multiple clusters, if you like :) - - - +--- +title: "Kubernetes on Azure with CoreOS and Weave" +--- + +Kubernetes on Azure with CoreOS and [Weave](http://weave.works) +--------------------------------------------------------------- + +{% include pagetoc.html %} + +- [Introduction](#introduction) +- [Prerequisites](#prerequisites) +- [Let's go!](#lets-go) +- [Deploying the workload](#deploying-the-workload) +- [Scaling](#scaling) +- [Exposing the app to the outside world](#exposing-the-app-to-the-outside-world) +- [Next steps](#next-steps) +- [Tear down...](#tear-down) + +## Introduction + +In this guide I will demonstrate how to deploy a Kubernetes cluster to Azure cloud. You will be using CoreOS with Weave, which implements simple and secure networking, in a transparent, yet robust way. The purpose of this guide is to provide an out-of-the-box implementation that can ultimately be taken into production with little change. It will demonstrate how to provision a dedicated Kubernetes master and etcd nodes, and show how to scale the cluster with ease. + +### Prerequisites + +1. You need an Azure account. + +## Let's go! + +To get started, you need to checkout the code: + +{% highlight sh %} +git clone https://github.com/kubernetes/kubernetes +cd kubernetes/docs/getting-started-guides/coreos/azure/ +{% endhighlight %} + +You will need to have [Node.js installed](http://nodejs.org/download/) on you machine. If you have previously used Azure CLI, you should have it already. + +First, you need to install some of the dependencies with + +{% highlight sh %} +npm install +{% endhighlight %} + +Now, all you need to do is: + +{% highlight sh %} +./azure-login.js -u +./create-kubernetes-cluster.js +{% endhighlight %} + +This script will provision a cluster suitable for production use, where there is a ring of 3 dedicated etcd nodes: 1 kubernetes master and 2 kubernetes nodes. The `kube-00` VM will be the master, your work loads are only to be deployed on the nodes, `kube-01` and `kube-02`. Initially, all VMs are single-core, to ensure a user of the free tier can reproduce it without paying extra. I will show how to add more bigger VMs later. + +![VMs in Azure](initial_cluster.png) + +Once the creation of Azure VMs has finished, you should see the following: + +{% highlight console %} +... +azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_1c1496016083b4_ssh_conf ` +azure_wrapper/info: The hosts in this deployment are: + [ 'etcd-00', 'etcd-01', 'etcd-02', 'kube-00', 'kube-01', 'kube-02' ] +azure_wrapper/info: Saved state into `./output/kube_1c1496016083b4_deployment.yml` +{% endhighlight %} + +Let's login to the master node like so: + +{% highlight sh %} +ssh -F ./output/kube_1c1496016083b4_ssh_conf kube-00 +{% endhighlight %} + +> Note: config file name will be different, make sure to use the one you see. + +Check there are 2 nodes in the cluster: + +{% highlight console %} +core@kube-00 ~ $ kubectl get nodes +NAME LABELS STATUS +kube-01 kubernetes.io/hostname=kube-01 Ready +kube-02 kubernetes.io/hostname=kube-02 Ready +{% endhighlight %} + +## Deploying the workload + +Let's follow the Guestbook example now: + +{% highlight sh %} +kubectl create -f ~/guestbook-example +{% endhighlight %} + +You need to wait for the pods to get deployed, run the following and wait for `STATUS` to change from `Pending` to `Running`. + +{% highlight sh %} +kubectl get pods --watch +{% endhighlight %} + +> Note: the most time it will spend downloading Docker container images on each of the nodes. + +Eventually you should see: + +{% highlight console %} +NAME READY STATUS RESTARTS AGE +frontend-0a9xi 1/1 Running 0 4m +frontend-4wahe 1/1 Running 0 4m +frontend-6l36j 1/1 Running 0 4m +redis-master-talmr 1/1 Running 0 4m +redis-slave-12zfd 1/1 Running 0 4m +redis-slave-3nbce 1/1 Running 0 4m +{% endhighlight %} + +## Scaling + +Two single-core nodes are certainly not enough for a production system of today. Let's scale the cluster by adding a couple of bigger nodes. + +You will need to open another terminal window on your machine and go to the same working directory (e.g. `~/Workspace/kubernetes/docs/getting-started-guides/coreos/azure/`). + +First, lets set the size of new VMs: + +{% highlight sh %} +export AZ_VM_SIZE=Large +{% endhighlight %} + +Now, run scale script with state file of the previous deployment and number of nodes to add: + +{% highlight console %} +core@kube-00 ~ $ ./scale-kubernetes-cluster.js ./output/kube_1c1496016083b4_deployment.yml 2 +... +azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_8f984af944f572_ssh_conf ` +azure_wrapper/info: The hosts in this deployment are: + [ 'etcd-00', + 'etcd-01', + 'etcd-02', + 'kube-00', + 'kube-01', + 'kube-02', + 'kube-03', + 'kube-04' ] +azure_wrapper/info: Saved state into `./output/kube_8f984af944f572_deployment.yml` +{% endhighlight %} + +> Note: this step has created new files in `./output`. + +Back on `kube-00`: + +{% highlight console %} +core@kube-00 ~ $ kubectl get nodes +NAME LABELS STATUS +kube-01 kubernetes.io/hostname=kube-01 Ready +kube-02 kubernetes.io/hostname=kube-02 Ready +kube-03 kubernetes.io/hostname=kube-03 Ready +kube-04 kubernetes.io/hostname=kube-04 Ready +{% endhighlight %} + +You can see that two more nodes joined happily. Let's scale the number of Guestbook instances now. + +First, double-check how many replication controllers there are: + +{% highlight console %} +core@kube-00 ~ $ kubectl get rc +ONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS +frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 3 +redis-master master redis name=redis-master 1 +redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 2 +{% endhighlight %} + +As there are 4 nodes, let's scale proportionally: + +{% highlight console %} +core@kube-00 ~ $ kubectl scale --replicas=4 rc redis-slave +>>>>>>> coreos/azure: Updates for 1.0 +scaled +core@kube-00 ~ $ kubectl scale --replicas=4 rc frontend +scaled +{% endhighlight %} + +Check what you have now: + +{% highlight console %} +core@kube-00 ~ $ kubectl get rc +CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS +frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 4 +redis-master master redis name=redis-master 1 +redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 4 +{% endhighlight %} + +You now will have more instances of front-end Guestbook apps and Redis slaves; and, if you look up all pods labeled `name=frontend`, you should see one running on each node. + +{% highlight console %} +core@kube-00 ~/guestbook-example $ kubectl get pods -l name=frontend +NAME READY STATUS RESTARTS AGE +frontend-0a9xi 1/1 Running 0 22m +frontend-4wahe 1/1 Running 0 22m +frontend-6l36j 1/1 Running 0 22m +frontend-z9oxo 1/1 Running 0 41s +{% endhighlight %} + +## Exposing the app to the outside world + +There is no native Azure load-balancer support in Kubernetes 1.0, however here is how you can expose the Guestbook app to the Internet. + +``` +./expose_guestbook_app_port.sh ./output/kube_1c1496016083b4_ssh_conf +Guestbook app is on port 31605, will map it to port 80 on kube-00 +info: Executing command vm endpoint create ++ Getting virtual machines ++ Reading network configuration ++ Updating network configuration +info: vm endpoint create command OK +info: Executing command vm endpoint show ++ Getting virtual machines +data: Name : tcp-80-31605 +data: Local port : 31605 +data: Protcol : tcp +data: Virtual IP Address : 137.117.156.164 +data: Direct server return : Disabled +info: vm endpoint show command OK +``` + +You then should be able to access it from anywhere via the Azure virtual IP for `kube-00` displayed above, i.e. `http://137.117.156.164/` in my case. + +## Next steps + +You now have a full-blow cluster running in Azure, congrats! + +You should probably try deploy other [example apps](../../../../examples/) or write your own ;) + +## Tear down... + +If you don't wish care about the Azure bill, you can tear down the cluster. It's easy to redeploy it, as you can see. + +{% highlight sh %} +./destroy-cluster.js ./output/kube_8f984af944f572_deployment.yml +{% endhighlight %} + +> Note: make sure to use the _latest state file_, as after scaling there is a new one. + +By the way, with the scripts shown, you can deploy multiple clusters, if you like :) + + + diff --git a/v1.1/docs/getting-started-guides/coreos/bare_metal_calico.md b/v1.1/docs/getting-started-guides/coreos/bare_metal_calico.md index ab8d913de6..5eb8e6ee4d 100644 --- a/v1.1/docs/getting-started-guides/coreos/bare_metal_calico.md +++ b/v1.1/docs/getting-started-guides/coreos/bare_metal_calico.md @@ -1,138 +1,124 @@ ---- -title: "Bare Metal CoreOS with Kubernetes and Project Calico" ---- - - -Bare Metal CoreOS with Kubernetes and Project Calico ------------------------------------------- -This guide explains how to deploy a bare-metal Kubernetes cluster on CoreOS using [Calico networking](http://www.projectcalico.org). - -Specifically, this guide will have you do the following: -- Deploy a Kubernetes master node on CoreOS using cloud-config -- Deploy two Kubernetes compute nodes with Calico Networking using cloud-config - -## Prerequisites - -1. At least three bare-metal machines (or VMs) to work with. This guide will configure them as follows - - 1 Kubernetes Master - - 2 Kubernetes Nodes -2. Your nodes should have IP connectivity. - -## Cloud-config - -This guide will use [cloud-config](https://coreos.com/docs/cluster-management/setup/cloudinit-cloud-config/) to configure each of the nodes in our Kubernetes cluster. - -For ease of distribution, the cloud-config files required for this demonstration can be found on [GitHub](https://github.com/projectcalico/calico-kubernetes-coreos-demo). - -This repo includes two cloud config files: -- `master-config.yaml`: Cloud-config for the Kubernetes master -- `node-config.yaml`: Cloud-config for each Kubernetes compute host - -In the next few steps you will be asked to configure these files and host them on an HTTP server where your cluster can access them. - -## Building Kubernetes - -To get the Kubernetes source, clone the GitHub repo, and build the binaries. - -``` -{% raw %} -git clone https://github.com/kubernetes/kubernetes.git -cd kubernetes -./build/release.sh -{% endraw %} -``` - -Once the binaries are built, host the entire `/_output/dockerized/bin///` folder on an accessible HTTP server so they can be accessed by the cloud-config. You'll point your cloud-config files at this HTTP server later. - -## Download CoreOS - -Let's download the CoreOS bootable ISO. We'll use this image to boot and install CoreOS on each server. - -``` -{% raw %} -wget http://stable.release.core-os.net/amd64-usr/current/coreos_production_iso_image.iso -{% endraw %} -``` - -You can also download the ISO from the [CoreOS website](https://coreos.com/docs/running-coreos/platforms/iso/). - -## Configure the Kubernetes Master - -Once you've downloaded the image, use it to boot your Kubernetes Master server. Once booted, you should be automatically logged in as the `core` user. - -Let's get the master-config.yaml and fill in the necessary variables. Run the following commands on your HTTP server to get the cloud-config files. - -``` -{% raw %} -git clone https://github.com/Metaswitch/calico-kubernetes-demo.git -cd calico-kubernetes-demo/coreos -{% endraw %} -``` - -You'll need to replace the following variables in the `master-config.yaml` file to match your deployment. -- ``: The public key you will use for SSH access to this server. -- ``: The address used to get the kubernetes binaries over HTTP. - -> **Note:** The config will prepend `"http://"` and append `"/(kubernetes | kubectl | ...)"` to your `KUBERNETES_LOC` variable:, format accordingly - -Host the modified `master-config.yaml` file and pull it on to your Kubernetes Master server. - -The CoreOS bootable ISO comes with a tool called `coreos-install` which will allow us to install CoreOS to disk and configure the install using cloud-config. The following command will download and install stable CoreOS, using the master-config.yaml file for configuration. - -``` -{% raw %} -sudo coreos-install -d /dev/sda -C stable -c master-config.yaml -{% endraw %} -``` - -Once complete, eject the bootable ISO and restart the server. When it comes back up, you should have SSH access as the `core` user using the public key provided in the master-config.yaml file. - -## Configure the compute hosts - ->The following steps will set up a Kubernetes node for use as a compute host. This demo uses two compute hosts, so you should run the following steps on each. - -First, boot up your node using the bootable ISO we downloaded earlier. You should be automatically logged in as the `core` user. - -Let's modify the `node-config.yaml` cloud-config file on your HTTP server. Make a copy for this node, and fill in the necessary variables. - -You'll need to replace the following variables in the `node-config.yaml` file to match your deployment. -- ``: Hostname for this node (e.g. kube-node1, kube-node2) -- ``: The public key you will use for SSH access to this server. -- ``: The IPv4 address of the Kubernetes master. -- ``: The address to use in order to get the kubernetes binaries over HTTP. -- ``: The IP and subnet to use for pods on this node. By default, this should fall within the 192.168.0.0/16 subnet. - -> Note: The DOCKER_BRIDGE_IP is the range used by this Kubernetes node to assign IP addresses to pods on this node. This subnet must not overlap with the subnets assigned to the other Kubernetes nodes in your cluster. Calico expects each DOCKER_BRIDGE_IP subnet to fall within 192.168.0.0/16 by default (e.g. 192.168.1.1/24 for node 1), but if you'd like to use pod IPs within a different subnet, simply run `calicoctl pool add ` and select DOCKER_BRIDGE_IP accordingly. - -Host the modified `node-config.yaml` file and pull it on to your Kubernetes node. - -``` -{% raw %} -wget http:///node-config.yaml -{% endraw %} -``` - -Install and configure CoreOS on the node using the following command. - -``` -{% raw %} -sudo coreos-install -d /dev/sda -C stable -c node-config.yaml -{% endraw %} -``` - -Once complete, restart the server. When it comes back up, you should have SSH access as the `core` user using the public key provided in the `node-config.yaml` file. It will take some time for the node to be fully configured. Once fully configured, you can check that the node is running with the following command on the Kubernetes master. - -``` -{% raw %} -/home/core/kubectl get nodes -{% endraw %} -``` - -## Testing the Cluster - -You should now have a functional bare-metal Kubernetes cluster with one master and two compute hosts. -Try running the [guestbook demo](../../../examples/guestbook/) to test out your new cluster! - - - - +--- +title: "Bare Metal CoreOS with Kubernetes and Project Calico" +--- + + +Bare Metal CoreOS with Kubernetes and Project Calico +------------------------------------------ +This guide explains how to deploy a bare-metal Kubernetes cluster on CoreOS using [Calico networking](http://www.projectcalico.org). + +Specifically, this guide will have you do the following: +- Deploy a Kubernetes master node on CoreOS using cloud-config +- Deploy two Kubernetes compute nodes with Calico Networking using cloud-config + +## Prerequisites + +1. At least three bare-metal machines (or VMs) to work with. This guide will configure them as follows + - 1 Kubernetes Master + - 2 Kubernetes Nodes +2. Your nodes should have IP connectivity. + +## Cloud-config + +This guide will use [cloud-config](https://coreos.com/docs/cluster-management/setup/cloudinit-cloud-config/) to configure each of the nodes in our Kubernetes cluster. + +For ease of distribution, the cloud-config files required for this demonstration can be found on [GitHub](https://github.com/projectcalico/calico-kubernetes-coreos-demo). + +This repo includes two cloud config files: +- `master-config.yaml`: Cloud-config for the Kubernetes master +- `node-config.yaml`: Cloud-config for each Kubernetes compute host + +In the next few steps you will be asked to configure these files and host them on an HTTP server where your cluster can access them. + +## Building Kubernetes + +To get the Kubernetes source, clone the GitHub repo, and build the binaries. + +``` +git clone https://github.com/kubernetes/kubernetes.git +cd kubernetes +./build/release.sh +``` + +Once the binaries are built, host the entire `/_output/dockerized/bin///` folder on an accessible HTTP server so they can be accessed by the cloud-config. You'll point your cloud-config files at this HTTP server later. + +## Download CoreOS + +Let's download the CoreOS bootable ISO. We'll use this image to boot and install CoreOS on each server. + +``` +wget http://stable.release.core-os.net/amd64-usr/current/coreos_production_iso_image.iso +``` + +You can also download the ISO from the [CoreOS website](https://coreos.com/docs/running-coreos/platforms/iso/). + +## Configure the Kubernetes Master + +Once you've downloaded the image, use it to boot your Kubernetes Master server. Once booted, you should be automatically logged in as the `core` user. + +Let's get the master-config.yaml and fill in the necessary variables. Run the following commands on your HTTP server to get the cloud-config files. + +``` +git clone https://github.com/Metaswitch/calico-kubernetes-demo.git +cd calico-kubernetes-demo/coreos +``` + +You'll need to replace the following variables in the `master-config.yaml` file to match your deployment. +- ``: The public key you will use for SSH access to this server. +- ``: The address used to get the kubernetes binaries over HTTP. + +> **Note:** The config will prepend `"http://"` and append `"/(kubernetes | kubectl | ...)"` to your `KUBERNETES_LOC` variable:, format accordingly + +Host the modified `master-config.yaml` file and pull it on to your Kubernetes Master server. + +The CoreOS bootable ISO comes with a tool called `coreos-install` which will allow us to install CoreOS to disk and configure the install using cloud-config. The following command will download and install stable CoreOS, using the master-config.yaml file for configuration. + +``` +sudo coreos-install -d /dev/sda -C stable -c master-config.yaml +``` + +Once complete, eject the bootable ISO and restart the server. When it comes back up, you should have SSH access as the `core` user using the public key provided in the master-config.yaml file. + +## Configure the compute hosts + +>The following steps will set up a Kubernetes node for use as a compute host. This demo uses two compute hosts, so you should run the following steps on each. + +First, boot up your node using the bootable ISO we downloaded earlier. You should be automatically logged in as the `core` user. + +Let's modify the `node-config.yaml` cloud-config file on your HTTP server. Make a copy for this node, and fill in the necessary variables. + +You'll need to replace the following variables in the `node-config.yaml` file to match your deployment. +- ``: Hostname for this node (e.g. kube-node1, kube-node2) +- ``: The public key you will use for SSH access to this server. +- ``: The IPv4 address of the Kubernetes master. +- ``: The address to use in order to get the kubernetes binaries over HTTP. +- ``: The IP and subnet to use for pods on this node. By default, this should fall within the 192.168.0.0/16 subnet. + +> Note: The DOCKER_BRIDGE_IP is the range used by this Kubernetes node to assign IP addresses to pods on this node. This subnet must not overlap with the subnets assigned to the other Kubernetes nodes in your cluster. Calico expects each DOCKER_BRIDGE_IP subnet to fall within 192.168.0.0/16 by default (e.g. 192.168.1.1/24 for node 1), but if you'd like to use pod IPs within a different subnet, simply run `calicoctl pool add ` and select DOCKER_BRIDGE_IP accordingly. + +Host the modified `node-config.yaml` file and pull it on to your Kubernetes node. + +``` +wget http:///node-config.yaml +``` + +Install and configure CoreOS on the node using the following command. + +``` +sudo coreos-install -d /dev/sda -C stable -c node-config.yaml +``` + +Once complete, restart the server. When it comes back up, you should have SSH access as the `core` user using the public key provided in the `node-config.yaml` file. It will take some time for the node to be fully configured. Once fully configured, you can check that the node is running with the following command on the Kubernetes master. + +``` +/home/core/kubectl get nodes +``` + +## Testing the Cluster + +You should now have a functional bare-metal Kubernetes cluster with one master and two compute hosts. +Try running the [guestbook demo](../../../examples/guestbook/) to test out your new cluster! + + + + diff --git a/v1.1/docs/getting-started-guides/coreos/bare_metal_offline.md b/v1.1/docs/getting-started-guides/coreos/bare_metal_offline.md index ad8bbe4d63..cbb4b8b24d 100644 --- a/v1.1/docs/getting-started-guides/coreos/bare_metal_offline.md +++ b/v1.1/docs/getting-started-guides/coreos/bare_metal_offline.md @@ -1,11 +1,8 @@ --- title: "Bare Metal CoreOS with Kubernetes (OFFLINE)" -section: guides --- Deploy a CoreOS running Kubernetes environment. This particular guild is made to help those in an OFFLINE system, wither for testing a POC before the real deal, or you are restricted to be totally offline for your applications. -## Table of Contents - {% include pagetoc.html %} ## Prerequisites @@ -35,7 +32,7 @@ Deploy a CoreOS running Kubernetes environment. This particular guild is made to ## Setup PXELINUX CentOS -To setup CentOS PXELINUX environment there is a complete [guide here](http://docs.fedoraproject.org/en-US/Fedora/7/html/Installation_Guide/ap-pxe-server.html). This section is the abbreviated version. +To setup CentOS PXELINUX environment there is a complete [guide here](http://docs.fedoraproject.org/en-US/Fedora/7/html/Installation_Guide/ap-pxe-server). This section is the abbreviated version. 1. Install packages needed on CentOS @@ -621,7 +618,7 @@ Reboot these servers to get the images PXEd and ready for running containers! Now that the CoreOS with Kubernetes installed is up and running lets spin up some Kubernetes pods to demonstrate the system. -See [a simple nginx example](../../../docs/user-guide/simple-nginx.html) to try out your new cluster. +See [a simple nginx example](/{{page.version}}/docs/user-guide/simple-nginx) to try out your new cluster. For more complete applications, please look in the [examples directory](../../../examples/). diff --git a/v1.1/docs/getting-started-guides/coreos/coreos_multinode_cluster.md b/v1.1/docs/getting-started-guides/coreos/coreos_multinode_cluster.md index 3a3cb7fda0..0fc6f8f409 100644 --- a/v1.1/docs/getting-started-guides/coreos/coreos_multinode_cluster.md +++ b/v1.1/docs/getting-started-guides/coreos/coreos_multinode_cluster.md @@ -1,10 +1,6 @@ --- title: "CoreOS Multinode Cluster" --- - - -# CoreOS Multinode Cluster - Use the [master.yaml](cloud-configs/master.yaml) and [node.yaml](cloud-configs/node.yaml) cloud-configs to provision a multi-node Kubernetes cluster. > **Attention**: This requires at least CoreOS version **[695.0.0][coreos695]**, which includes `etcd2`. @@ -25,16 +21,16 @@ Use the [master.yaml](cloud-configs/master.yaml) and [node.yaml](cloud-configs/n #### Provision the Master {% highlight sh %} -{% raw %} + aws ec2 create-security-group --group-name kubernetes --description "Kubernetes Security Group" aws ec2 authorize-security-group-ingress --group-name kubernetes --protocol tcp --port 22 --cidr 0.0.0.0/0 aws ec2 authorize-security-group-ingress --group-name kubernetes --protocol tcp --port 80 --cidr 0.0.0.0/0 aws ec2 authorize-security-group-ingress --group-name kubernetes --source-security-group-name kubernetes -{% endraw %} + {% endhighlight %} {% highlight sh %} -{% raw %} + aws ec2 run-instances \ --image-id \ --key-name \ @@ -42,15 +38,15 @@ aws ec2 run-instances \ --security-groups kubernetes \ --instance-type m3.medium \ --user-data file://master.yaml -{% endraw %} + {% endhighlight %} #### Capture the private IP address {% highlight sh %} -{% raw %} + aws ec2 describe-instances --instance-id -{% endraw %} + {% endhighlight %} #### Edit node.yaml @@ -60,7 +56,7 @@ Edit `node.yaml` and replace all instances of `` with the pri #### Provision worker nodes {% highlight sh %} -{% raw %} + aws ec2 run-instances \ --count 1 \ --image-id \ @@ -69,7 +65,7 @@ aws ec2 run-instances \ --security-groups kubernetes \ --instance-type m3.medium \ --user-data file://node.yaml -{% endraw %} + {% endhighlight %} ### Google Compute Engine (GCE) @@ -79,7 +75,7 @@ aws ec2 run-instances \ #### Provision the Master {% highlight sh %} -{% raw %} + gcloud compute instances create master \ --image-project coreos-cloud \ --image \ @@ -87,15 +83,15 @@ gcloud compute instances create master \ --machine-type n1-standard-1 \ --zone us-central1-a \ --metadata-from-file user-data=master.yaml -{% endraw %} + {% endhighlight %} #### Capture the private IP address {% highlight sh %} -{% raw %} + gcloud compute instances list -{% endraw %} + {% endhighlight %} #### Edit node.yaml @@ -105,7 +101,7 @@ Edit `node.yaml` and replace all instances of `` with the pri #### Provision worker nodes {% highlight sh %} -{% raw %} + gcloud compute instances create node1 \ --image-project coreos-cloud \ --image \ @@ -113,7 +109,7 @@ gcloud compute instances create node1 \ --machine-type n1-standard-1 \ --zone us-central1-a \ --metadata-from-file user-data=node.yaml -{% endraw %} + {% endhighlight %} #### Establish network connectivity @@ -132,51 +128,51 @@ These instructions were tested on the Ice House release on a Metacloud distribut Make sure the environment variables are set for OpenStack such as: {% highlight sh %} -{% raw %} + OS_TENANT_ID OS_PASSWORD OS_AUTH_URL OS_USERNAME OS_TENANT_NAME -{% endraw %} + {% endhighlight %} Test this works with something like: ``` -{% raw %} + nova list -{% endraw %} + ``` #### Get a Suitable CoreOS Image -You'll need a [suitable version of CoreOS image for OpenStack](https://coreos.com/os/docs/latest/booting-on-openstack.html) +You'll need a [suitable version of CoreOS image for OpenStack](https://coreos.com/os/docs/latest/booting-on-openstack) Once you download that, upload it to glance. An example is shown below: {% highlight sh %} -{% raw %} + glance image-create --name CoreOS723 \ --container-format bare --disk-format qcow2 \ --file coreos_production_openstack_image.img \ --is-public True -{% endraw %} + {% endhighlight %} #### Create security group {% highlight sh %} -{% raw %} + nova secgroup-create kubernetes "Kubernetes Security Group" nova secgroup-add-rule kubernetes tcp 22 22 0.0.0.0/0 nova secgroup-add-rule kubernetes tcp 80 80 0.0.0.0/0 -{% endraw %} + {% endhighlight %} #### Provision the Master {% highlight sh %} -{% raw %} + nova boot \ --image \ --key-name \ @@ -184,7 +180,7 @@ nova boot \ --security-group kubernetes \ --user-data files/master.yaml \ kube-master -{% endraw %} + {% endhighlight %} `````` is the CoreOS image name. In our example we can use the image we created in the previous step and put in 'CoreOS723' @@ -198,17 +194,17 @@ The important part is to ensure you have the files/master.yml as this is what wi Next, assign it a public IP address: ``` -{% raw %} + nova floating-ip-list -{% endraw %} + ``` Get an IP address that's free and run: ``` -{% raw %} + nova floating-ip-associate kube-master -{% endraw %} + ``` where `````` is the IP address that was available from the ```nova floating-ip-list``` command. @@ -218,7 +214,7 @@ where `````` is the IP address that was available from the ```nova f Edit ```node.yaml``` and replace all instances of `````` with the private IP address of the master node. You can get this by running ```nova show kube-master``` assuming you named your instance kube master. This is not the floating IP address you just assigned it. {% highlight sh %} -{% raw %} + nova boot \ --image \ --key-name \ @@ -226,7 +222,7 @@ nova boot \ --security-group kubernetes \ --user-data files/node.yaml \ minion01 -{% endraw %} + {% endhighlight %} This is basically the same as the master nodes but with the node.yaml post-boot script instead of the master. diff --git a/v1.1/docs/getting-started-guides/dcos.md b/v1.1/docs/getting-started-guides/dcos.md index 6c18753f75..95cb05ce12 100644 --- a/v1.1/docs/getting-started-guides/dcos.md +++ b/v1.1/docs/getting-started-guides/dcos.md @@ -1,175 +1,138 @@ ---- -title: "Getting started with Kubernetes on DCOS" ---- - - -Getting started with Kubernetes on DCOS ----------------------------------------- - -This guide will walk you through installing [Kubernetes-Mesos](https://github.com/mesosphere/kubernetes-mesos) on [Datacenter Operating System (DCOS)](https://mesosphere.com/product/) with the [DCOS CLI](https://github.com/mesosphere/dcos-cli) and operating Kubernetes with the [DCOS Kubectl plugin](https://github.com/mesosphere/dcos-kubectl). - -**Table of Contents** - - - - [About Kubernetes on DCOS](#about-kubernetes-on-dcos) - - [Resources](#resources) - - [Prerequisites](#prerequisites) - - [Install](#install) - - [Uninstall](#uninstall) - - - - -## About Kubernetes on DCOS - -DCOS is system software that manages computer cluster hardware and software resources and provides common services for distributed applications. Among other services, it provides [Apache Mesos](http://mesos.apache.org/) as its cluster kernel and [Marathon](https://mesosphere.github.io/marathon/) as its init system. With DCOS CLI, Mesos frameworks like [Kubernetes-Mesos](https://github.com/mesosphere/kubernetes-mesos) can be installed with a single command. - -Another feature of the DCOS CLI is that it allows plugins like the [DCOS Kubectl plugin](https://github.com/mesosphere/dcos-kubectl). This allows for easy access to a version-compatible Kubectl without having to manually download or install. - -Further information about the benefits of installing Kubernetes on DCOS can be found in the [Kubernetes-Mesos documentation](https://releases.k8s.io/release-1.1/contrib/mesos/README.md). - -For more details about the Kubernetes DCOS packaging, see the [Kubernetes-Mesos project](https://github.com/mesosphere/kubernetes-mesos). - -Since Kubernetes-Mesos is still alpha, it is a good idea to familiarize yourself with the [current known issues](https://releases.k8s.io/release-1.1/contrib/mesos/docs/issues.md) which may limit or modify the behavior of Kubernetes on DCOS. - -If you have problems completing the steps below, please [file an issue against the kubernetes-mesos project](https://github.com/mesosphere/kubernetes-mesos/issues). - - -## Resources - -Explore the following resources for more information about Kubernetes, Kubernetes on Mesos/DCOS, and DCOS itself. - -- [DCOS Documentation](https://docs.mesosphere.com/) -- [Managing DCOS Services](https://docs.mesosphere.com/services/kubernetes/) -- [Kubernetes Examples](../../examples/README.html) -- [Kubernetes on Mesos Documentation](https://releases.k8s.io/release-1.1/contrib/mesos/README.md) -- [Kubernetes on Mesos Release Notes](https://github.com/mesosphere/kubernetes-mesos/releases) -- [Kubernetes on DCOS Package Source](https://github.com/mesosphere/kubernetes-mesos) - - -## Prerequisites - -- A running [DCOS cluster](https://mesosphere.com/product/) - - [DCOS Community Edition](https://docs.mesosphere.com/install/) is currently available on [AWS](https://mesosphere.com/amazon/). - - [DCOS Enterprise Edition](https://mesosphere.com/product/) can be deployed on virtual or bare metal machines. Contact sales@mesosphere.com for more info and to set up an engagement. -- [DCOS CLI](https://docs.mesosphere.com/install/cli/) installed locally - - -## Install - -1. Configure and validate the [Mesosphere Multiverse](https://github.com/mesosphere/multiverse) as a package source repository - - ``` -{% raw %} - $ dcos config prepend package.sources https://github.com/mesosphere/multiverse/archive/version-1.x.zip - $ dcos package update --validate -{% endraw %} - ``` - -2. Install etcd - - By default, the Kubernetes DCOS package starts a single-node etcd. In order to avoid state loss in the event of Kubernetes component container failure, install an HA [etcd-mesos](https://github.com/mesosphere/etcd-mesos) cluster on DCOS. - - ``` -{% raw %} - $ dcos package install etcd -{% endraw %} - ``` - -3. Verify that etcd is installed and healthy - - The etcd cluster takes a short while to deploy. Verify that `/etcd` is healthy before going on to the next step. - - ``` -{% raw %} - $ dcos marathon app list - ID MEM CPUS TASKS HEALTH DEPLOYMENT CONTAINER CMD - /etcd 128 0.2 1/1 1/1 --- DOCKER None -{% endraw %} - ``` - -4. Create Kubernetes installation configuration - - Configure Kubernetes to use the HA etcd installed on DCOS. - - ``` -{% raw %} - $ cat >/tmp/options.json </tmp/options.json < -{% endraw %} -``` - -Otherwise, we'll use latest `hyperkube` image as default k8s version. - -## Master Node - -The first step in the process is to initialize the master node. - -Clone the Kubernetes repo, and run [master.sh](docker-multinode/master.sh) on the master machine with root: - -{% highlight sh %} -{% raw %} -cd kubernetes/docs/getting-started-guides/docker-multinode/ -./master.sh -{% endraw %} -{% endhighlight %} - -`Master done!` - -See [here](docker-multinode/master.html) for detailed instructions explanation. - -## Adding a worker node - -Once your master is up and running you can add one or more workers on different machines. - -Clone the Kubernetes repo, and run [worker.sh](docker-multinode/worker.sh) on the worker machine with root: - -{% highlight sh %} -{% raw %} -export MASTER_IP= -cd kubernetes/docs/getting-started-guides/docker-multinode/ -./worker.sh -{% endraw %} -{% endhighlight %} - -`Worker done!` - -See [here](docker-multinode/worker.html) for detailed instructions explanation. - -## Deploy a DNS - -See [here](docker-multinode/deployDNS.html) for instructions. - -## Testing your cluster - -Once your cluster has been created you can [test it out](docker-multinode/testing.html) - +--- +title: "Running Multi-Node Kubernetes Using Docker" +--- + +_Note_: +These instructions are somewhat significantly more advanced than the [single node](docker) instructions. If you are +interested in just starting to explore Kubernetes, we recommend that you start there. + +_Note_: +There is a [bug](https://github.com/docker/docker/issues/14106) in Docker 1.7.0 that prevents this from working correctly. +Please install Docker 1.6.2 or Docker 1.7.1. + + + +{% include pagetoc.html %} + +## Prerequisites + +1. You need a machine with docker of right version installed. + +## Overview + +This guide will set up a 2-node Kubernetes cluster, consisting of a _master_ node which hosts the API server and orchestrates work +and a _worker_ node which receives work from the master. You can repeat the process of adding worker nodes an arbitrary number of +times to create larger clusters. + +Here's a diagram of what the final result will look like: +![Kubernetes Single Node on Docker](k8s-docker.png) + +### Bootstrap Docker + +This guide also uses a pattern of running two instances of the Docker daemon + 1) A _bootstrap_ Docker instance which is used to start system daemons like `flanneld` and `etcd` + 2) A _main_ Docker instance which is used for the Kubernetes infrastructure and user's scheduled containers + +This pattern is necessary because the `flannel` daemon is responsible for setting up and managing the network that interconnects +all of the Docker containers created by Kubernetes. To achieve this, it must run outside of the _main_ Docker daemon. However, +it is still useful to use containers for deployment and management, so we create a simpler _bootstrap_ daemon to achieve this. + +You can specify k8s version on very node before install: + +``` +export K8S_VERSION= +``` + +Otherwise, we'll use latest `hyperkube` image as default k8s version. + +## Master Node + +The first step in the process is to initialize the master node. + +Clone the Kubernetes repo, and run [master.sh](/{{page.version}}/docs/getting-started-guides/docker-multinode/master.sh) on the master machine with root: + +{% highlight sh %} +cd kubernetes/docs/getting-started-guides/docker-multinode/ +./master.sh +{% endhighlight %} + +`Master done!` + +See [here](/{{page.version}}/docs/getting-started-guides/docker-multinode/master) for detailed instructions explanation. + +## Adding a worker node + +Once your master is up and running you can add one or more workers on different machines. + +Clone the Kubernetes repo, and run [worker.sh](/{{page.version}}/docs/getting-started-guides/docker-multinode/worker.sh) on the worker machine with root: + +{% highlight sh %} +export MASTER_IP= +cd kubernetes/docs/getting-started-guides/docker-multinode/ +./worker.sh +{% endhighlight %} + +`Worker done!` + +See [here](/{{page.version}}/docs/getting-started-guides/docker-multinode/worker) for detailed instructions explanation. + +## Deploy a DNS + +See [here](/{{page.version}}/docs/getting-started-guides/docker-multinode/deployDNS) for instructions. + +## Testing your cluster + +Once your cluster has been created you can [test it out](/{{page.version}}/docs/getting-started-guides/docker-multinode/testing) + For more complete applications, please look in the [examples directory](../../examples/) \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/docker-multinode/deployDNS.md b/v1.1/docs/getting-started-guides/docker-multinode/deployDNS.md index 5335fc385a..20599b7ba4 100644 --- a/v1.1/docs/getting-started-guides/docker-multinode/deployDNS.md +++ b/v1.1/docs/getting-started-guides/docker-multinode/deployDNS.md @@ -1,10 +1,6 @@ --- title: "Deploy DNS" --- - - -## Deploy DNS - ### Get the template file First of all, download the template dns rc and svc file from @@ -18,7 +14,7 @@ First of all, download the template dns rc and svc file from Then you need to set `DNS_REPLICAS` , `DNS_DOMAIN` , `DNS_SERVER_IP` , `KUBE_SERVER` ENV. ``` -{% raw %} + $ export DNS_REPLICAS=1 $ export DNS_DOMAIN=cluster.local # specify in startup parameter `--cluster-domain` for containerized kubelet @@ -26,28 +22,28 @@ $ export DNS_DOMAIN=cluster.local # specify in startup parameter `--cluster-doma $ export DNS_SERVER_IP=10.0.0.10 # specify in startup parameter `--cluster-dns` for containerized kubelet $ export KUBE_SERVER=10.10.103.250 # your master server ip, you may change it -{% endraw %} + ``` ### Replace the corresponding value in the template. ``` -{% raw %} + $ sed -e "s/{{ pillar\['dns_replicas'\] }}/${DNS_REPLICAS}/g;s/{{ pillar\['dns_domain'\] }}/${DNS_DOMAIN}/g;s/{kube_server_url}/${KUBE_SERVER}/g;" skydns-rc.yaml.in > ./skydns-rc.yaml $ sed -e "s/{{ pillar\['dns_server'\] }}/${DNS_SERVER_IP}/g" skydns-svc.yaml.in > ./skydns-svc.yaml -{% endraw %} + ``` ### Use `kubectl` to create skydns rc and service ``` -{% raw %} + $ kubectl -s "$KUBE_SERVER:8080" --namespace=kube-system create -f ./skydns-rc.yaml $ kubectl -s "$KUBE_SERVER:8080" --namespace=kube-system create -f ./skydns-svc.yaml -{% endraw %} + ``` ### Test if DNS works diff --git a/v1.1/docs/getting-started-guides/docker-multinode/master.md b/v1.1/docs/getting-started-guides/docker-multinode/master.md index e416c3338e..249790ec8d 100644 --- a/v1.1/docs/getting-started-guides/docker-multinode/master.md +++ b/v1.1/docs/getting-started-guides/docker-multinode/master.md @@ -1,10 +1,6 @@ --- title: "Installing a Kubernetes Master Node via Docker" --- - - -## Installing a Kubernetes Master Node via Docker - We'll begin by setting up the master node. For the purposes of illustration, we'll assume that the IP of this machine is `${MASTER_IP}` There are two main phases to installing the master: @@ -27,9 +23,9 @@ Docker containers themselves. To achieve this, we need a separate "bootstrap" i Run: {% highlight sh %} -{% raw %} + sudo sh -c 'docker -d -H unix:///var/run/docker-bootstrap.sock -p /var/run/docker-bootstrap.pid --iptables=false --ip-masq=false --bridge=none --graph=/var/lib/docker-bootstrap 2> /var/log/docker-bootstrap.log 1> /dev/null &' -{% endraw %} + {% endhighlight %} _Important Note_: @@ -42,17 +38,17 @@ across reboots and failures. Run: {% highlight sh %} -{% raw %} + sudo docker -H unix:///var/run/docker-bootstrap.sock run --net=host -d gcr.io/google_containers/etcd:2.0.12 /usr/local/bin/etcd --addr=127.0.0.1:4001 --bind-addr=0.0.0.0:4001 --data-dir=/var/etcd/data -{% endraw %} + {% endhighlight %} Next, you need to set a CIDR range for flannel. This CIDR should be chosen to be non-overlapping with any existing network you are using: {% highlight sh %} -{% raw %} + sudo docker -H unix:///var/run/docker-bootstrap.sock run --net=host gcr.io/google_containers/etcd:2.0.12 etcdctl set /coreos.com/network/config '{ "Network": "10.1.0.0/16" }' -{% endraw %} + {% endhighlight %} @@ -69,17 +65,17 @@ To re-configure Docker to use flannel, we need to take docker down, run flannel Turning down Docker is system dependent, it may be: {% highlight sh %} -{% raw %} + sudo /etc/init.d/docker stop -{% endraw %} + {% endhighlight %} or {% highlight sh %} -{% raw %} + sudo systemctl stop docker -{% endraw %} + {% endhighlight %} or it may be something else. @@ -89,9 +85,9 @@ or it may be something else. Now run flanneld itself: {% highlight sh %} -{% raw %} + sudo docker -H unix:///var/run/docker-bootstrap.sock run -d --net=host --privileged -v /dev/net:/dev/net quay.io/coreos/flannel:0.5.0 -{% endraw %} + {% endhighlight %} The previous command should have printed a really long hash, copy this hash. @@ -99,9 +95,9 @@ The previous command should have printed a really long hash, copy this hash. Now get the subnet settings from flannel: {% highlight sh %} -{% raw %} + sudo docker -H unix:///var/run/docker-bootstrap.sock exec cat /run/flannel/subnet.env -{% endraw %} + {% endhighlight %} #### Edit the docker configuration @@ -113,9 +109,9 @@ This may be in `/etc/default/docker` or `/etc/systemd/service/docker.service` or Regardless, you need to add the following to the docker command line: {% highlight sh %} -{% raw %} + --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} -{% endraw %} + {% endhighlight %} #### Remove the existing Docker bridge @@ -123,10 +119,10 @@ Regardless, you need to add the following to the docker command line: Docker creates a bridge named `docker0` by default. You need to remove this: {% highlight sh %} -{% raw %} + sudo /sbin/ifconfig docker0 down sudo brctl delbr docker0 -{% endraw %} + {% endhighlight %} You may need to install the `bridge-utils` package for the `brctl` binary. @@ -136,17 +132,17 @@ You may need to install the `bridge-utils` package for the `brctl` binary. Again this is system dependent, it may be: {% highlight sh %} -{% raw %} + sudo /etc/init.d/docker start -{% endraw %} + {% endhighlight %} it may be: {% highlight sh %} -{% raw %} + systemctl start docker -{% endraw %} + {% endhighlight %} ## Starting the Kubernetes Master @@ -154,7 +150,7 @@ systemctl start docker Ok, now that your networking is set up, you can startup Kubernetes, this is the same as the single-node case, we will use the "main" instance of the Docker daemon for the Kubernetes components. {% highlight sh %} -{% raw %} + sudo docker run \ --volume=/:/rootfs:ro \ --volume=/sys:/sys:ro \ @@ -167,7 +163,7 @@ sudo docker run \ --pid=host \ -d \ gcr.io/google_containers/hyperkube:v1.0.1 /hyperkube kubelet --api-servers=http://localhost:8080 --v=2 --address=0.0.0.0 --enable-server --hostname-override=127.0.0.1 --config=/etc/kubernetes/manifests-multi --cluster-dns=10.0.0.10 --cluster-domain=cluster.local -{% endraw %} + {% endhighlight %} > Note that `--cluster-dns` and `--cluster-domain` is used to deploy dns, feel free to discard them if dns is not needed. @@ -175,9 +171,9 @@ sudo docker run \ ### Also run the service proxy {% highlight sh %} -{% raw %} + sudo docker run -d --net=host --privileged gcr.io/google_containers/hyperkube:v1.0.1 /hyperkube proxy --master=http://127.0.0.1:8080 --v=2 -{% endraw %} + {% endhighlight %} ### Test it out @@ -191,18 +187,18 @@ Download the kubectl binary and make it available by editing your PATH ENV. List the nodes {% highlight sh %} -{% raw %} + kubectl get nodes -{% endraw %} + {% endhighlight %} This should print: {% highlight console %} -{% raw %} + NAME LABELS STATUS 127.0.0.1 kubernetes.io/hostname=127.0.0.1 Ready -{% endraw %} + {% endhighlight %} If the status of the node is `NotReady` or `Unknown` please check that all of the containers you created are successfully running. @@ -211,7 +207,7 @@ If all else fails, ask questions on [Slack](../../troubleshooting.html#slack). ### Next steps -Move on to [adding one or more workers](worker.html) or [deploy a dns](deployDNS.html) +Move on to [adding one or more workers](worker) or [deploy a dns](deployDNS) diff --git a/v1.1/docs/getting-started-guides/docker-multinode/testing.md b/v1.1/docs/getting-started-guides/docker-multinode/testing.md index effa9a5e05..94db9e9c5d 100644 --- a/v1.1/docs/getting-started-guides/docker-multinode/testing.md +++ b/v1.1/docs/getting-started-guides/docker-multinode/testing.md @@ -1,26 +1,22 @@ --- title: "Testing your Kubernetes cluster." --- - - -## Testing your Kubernetes cluster. - To validate that your node(s) have been added, run: {% highlight sh %} -{% raw %} + kubectl get nodes -{% endraw %} + {% endhighlight %} That should show something like: {% highlight console %} -{% raw %} + NAME LABELS STATUS 10.240.99.26 kubernetes.io/hostname=10.240.99.26 Ready 127.0.0.1 kubernetes.io/hostname=127.0.0.1 Ready -{% endraw %} + {% endhighlight %} If the status of any node is `Unknown` or `NotReady` your cluster is broken, double check that all containers are running properly, and if all else fails, contact us on [Slack](../../troubleshooting.html#slack). @@ -28,9 +24,9 @@ If the status of any node is `Unknown` or `NotReady` your cluster is broken, dou ### Run an application {% highlight sh %} -{% raw %} + kubectl -s http://localhost:8080 run nginx --image=nginx --port=80 -{% endraw %} + {% endhighlight %} now run `docker ps` you should see nginx running. You may need to wait a few minutes for the image to get pulled. @@ -38,33 +34,33 @@ now run `docker ps` you should see nginx running. You may need to wait a few mi ### Expose it as a service {% highlight sh %} -{% raw %} + kubectl expose rc nginx --port=80 -{% endraw %} + {% endhighlight %} Run the following command to obtain the IP of this service we just created. There are two IPs, the first one is internal (CLUSTER_IP), and the second one is the external load-balanced IP. {% highlight sh %} -{% raw %} + kubectl get svc nginx -{% endraw %} + {% endhighlight %} Alternatively, you can obtain only the first IP (CLUSTER_IP) by running: {% highlight sh %} -{% raw %} + kubectl get svc nginx --template={{.spec.clusterIP}} -{% endraw %} + {% endhighlight %} Hit the webserver with the first IP (CLUSTER_IP): {% highlight sh %} -{% raw %} + curl -{% endraw %} + {% endhighlight %} Note that you will need run this curl command on your boot2docker VM if you are running on OS X. @@ -74,17 +70,17 @@ Note that you will need run this curl command on your boot2docker VM if you are Now try to scale up the nginx you created before: {% highlight sh %} -{% raw %} + kubectl scale rc nginx --replicas=3 -{% endraw %} + {% endhighlight %} And list the pods {% highlight sh %} -{% raw %} + kubectl get pods -{% endraw %} + {% endhighlight %} You should see pods landing on the newly added machine. diff --git a/v1.1/docs/getting-started-guides/docker-multinode/worker.md b/v1.1/docs/getting-started-guides/docker-multinode/worker.md index 73fdee0863..d1066d7b3d 100644 --- a/v1.1/docs/getting-started-guides/docker-multinode/worker.md +++ b/v1.1/docs/getting-started-guides/docker-multinode/worker.md @@ -1,15 +1,9 @@ --- title: "Adding a Kubernetes worker node via Docker." --- - - -## Adding a Kubernetes worker node via Docker. - - - These instructions are very similar to the master set-up above, but they are duplicated for clarity. You need to repeat these instructions for each node you want to join the cluster. -We will assume that the IP address of this node is `${NODE_IP}` and you have the IP address of the master in `${MASTER_IP}` that you created in the [master instructions](master.html). +We will assume that the IP address of this node is `${NODE_IP}` and you have the IP address of the master in `${MASTER_IP}` that you created in the [master instructions](master). For each worker node, there are three steps: * [Set up `flanneld` on the worker node](#set-up-flanneld-on-the-worker-node) @@ -32,9 +26,9 @@ As previously, we need a second instance of the Docker daemon running to bootstr Run: {% highlight sh %} -{% raw %} + sudo sh -c 'docker -d -H unix:///var/run/docker-bootstrap.sock -p /var/run/docker-bootstrap.pid --iptables=false --ip-masq=false --bridge=none --graph=/var/lib/docker-bootstrap 2> /var/log/docker-bootstrap.log 1> /dev/null &' -{% endraw %} + {% endhighlight %} _Important Note_: @@ -48,17 +42,17 @@ To re-configure Docker to use flannel, we need to take docker down, run flannel Turning down Docker is system dependent, it may be: {% highlight sh %} -{% raw %} + sudo /etc/init.d/docker stop -{% endraw %} + {% endhighlight %} or {% highlight sh %} -{% raw %} + sudo systemctl stop docker -{% endraw %} + {% endhighlight %} or it may be something else. @@ -68,9 +62,9 @@ or it may be something else. Now run flanneld itself, this call is slightly different from the above, since we point it at the etcd instance on the master. {% highlight sh %} -{% raw %} + sudo docker -H unix:///var/run/docker-bootstrap.sock run -d --net=host --privileged -v /dev/net:/dev/net quay.io/coreos/flannel:0.5.0 /opt/bin/flanneld --etcd-endpoints=http://${MASTER_IP}:4001 -{% endraw %} + {% endhighlight %} The previous command should have printed a really long hash, copy this hash. @@ -78,9 +72,9 @@ The previous command should have printed a really long hash, copy this hash. Now get the subnet settings from flannel: {% highlight sh %} -{% raw %} + sudo docker -H unix:///var/run/docker-bootstrap.sock exec cat /run/flannel/subnet.env -{% endraw %} + {% endhighlight %} @@ -93,9 +87,9 @@ This may be in `/etc/default/docker` or `/etc/systemd/service/docker.service` or Regardless, you need to add the following to the docker command line: {% highlight sh %} -{% raw %} + --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} -{% endraw %} + {% endhighlight %} #### Remove the existing Docker bridge @@ -103,10 +97,10 @@ Regardless, you need to add the following to the docker command line: Docker creates a bridge named `docker0` by default. You need to remove this: {% highlight sh %} -{% raw %} + sudo /sbin/ifconfig docker0 down sudo brctl delbr docker0 -{% endraw %} + {% endhighlight %} You may need to install the `bridge-utils` package for the `brctl` binary. @@ -116,17 +110,17 @@ You may need to install the `bridge-utils` package for the `brctl` binary. Again this is system dependent, it may be: {% highlight sh %} -{% raw %} + sudo /etc/init.d/docker start -{% endraw %} + {% endhighlight %} it may be: {% highlight sh %} -{% raw %} + systemctl start docker -{% endraw %} + {% endhighlight %} ### Start Kubernetes on the worker node @@ -136,7 +130,7 @@ systemctl start docker Again this is similar to the above, but the `--api-servers` now points to the master we set up in the beginning. {% highlight sh %} -{% raw %} + sudo docker run \ --volume=/:/rootfs:ro \ --volume=/sys:/sys:ro \ @@ -149,7 +143,7 @@ sudo docker run \ --pid=host \ -d \ gcr.io/google_containers/hyperkube:v1.0.1 /hyperkube kubelet --api-servers=http://${MASTER_IP}:8080 --v=2 --address=0.0.0.0 --enable-server --hostname-override=$(hostname -i) --cluster-dns=10.0.0.10 --cluster-domain=cluster.local -{% endraw %} + {% endhighlight %} #### Run the service proxy @@ -157,14 +151,14 @@ sudo docker run \ The service proxy provides load-balancing between groups of containers defined by Kubernetes `Services` {% highlight sh %} -{% raw %} + sudo docker run -d --net=host --privileged gcr.io/google_containers/hyperkube:v1.0.1 /hyperkube proxy --master=http://${MASTER_IP}:8080 --v=2 -{% endraw %} + {% endhighlight %} ### Next steps -Move on to [testing your cluster](testing.html) or [add another node](#adding-a-kubernetes-worker-node-via-docker) +Move on to [testing your cluster](testing) or [add another node](#adding-a-kubernetes-worker-node-via-docker) diff --git a/v1.1/docs/getting-started-guides/docker.md b/v1.1/docs/getting-started-guides/docker.md index 3c09dc2ce2..9216849d17 100644 --- a/v1.1/docs/getting-started-guides/docker.md +++ b/v1.1/docs/getting-started-guides/docker.md @@ -1,177 +1,148 @@ ---- -title: "Running Kubernetes locally via Docker" -section: guides ---- - -## Overview - -The following instructions show you how to set up a simple, single node Kubernetes cluster using Docker. - -Here's a diagram of what the final result will look like: - -![Kubernetes Single Node on Docker](/{{ page.version }}/docs/getting-started-guides/k8s-singlenode-docker.png) - -## Table of Contents - -{% include pagetoc.html %} - -## Prerequisites - -1. You need to have docker installed on one machine. -2. Your kernel should support memory and swap accounting. Ensure that the -following configs are turned on in your linux kernel: - -{% highlight console %} -{% raw %} - CONFIG_RESOURCE_COUNTERS=y - CONFIG_MEMCG=y - CONFIG_MEMCG_SWAP=y - CONFIG_MEMCG_SWAP_ENABLED=y - CONFIG_MEMCG_KMEM=y -{% endraw %} -{% endhighlight %} - -3. Enable the memory and swap accounting in the kernel, at boot, as command line -parameters as follows: - -{% highlight console %} -{% raw %} - GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1" -{% endraw %} -{% endhighlight %} - - NOTE: The above is specifically for GRUB2. - You can check the command line parameters passed to your kernel by looking at the - output of /proc/cmdline: - -{% highlight console %} -{% raw %} - $cat /proc/cmdline - BOOT_IMAGE=/boot/vmlinuz-3.18.4-aufs root=/dev/sda5 ro cgroup_enable=memory - swapaccount=1 -{% endraw %} -{% endhighlight %} - -### Step One: Run etcd - -{% highlight sh %} -{% raw %} -docker run --net=host -d gcr.io/google_containers/etcd:2.0.12 /usr/local/bin/etcd --addr=127.0.0.1:4001 --bind-addr=0.0.0.0:4001 --data-dir=/var/etcd/data -{% endraw %} -{% endhighlight %} - -### Step Two: Run the master - -{% highlight sh %} -{% raw %} -docker run \ - --volume=/:/rootfs:ro \ - --volume=/sys:/sys:ro \ - --volume=/dev:/dev \ - --volume=/var/lib/docker/:/var/lib/docker:ro \ - --volume=/var/lib/kubelet/:/var/lib/kubelet:rw \ - --volume=/var/run:/var/run:rw \ - --net=host \ - --pid=host \ - --privileged=true \ - -d \ - gcr.io/google_containers/hyperkube:v1.0.1 \ - /hyperkube kubelet --containerized --hostname-override="127.0.0.1" --address="0.0.0.0" --api-servers=http://localhost:8080 --config=/etc/kubernetes/manifests -{% endraw %} -{% endhighlight %} - -This actually runs the kubelet, which in turn runs a [pod](../user-guide/pods.html) that contains the other master components. - -### Step Three: Run the service proxy - -{% highlight sh %} -{% raw %} -docker run -d --net=host --privileged gcr.io/google_containers/hyperkube:v1.0.1 /hyperkube proxy --master=http://127.0.0.1:8080 --v=2 -{% endraw %} -{% endhighlight %} - -### Test it out - -At this point you should have a running Kubernetes cluster. You can test this by downloading the kubectl -binary -([OS X](https://storage.googleapis.com/kubernetes-release/release/v1.0.1/bin/darwin/amd64/kubectl)) -([linux](https://storage.googleapis.com/kubernetes-release/release/v1.0.1/bin/linux/amd64/kubectl)) - -*Note:* -On OS/X you will need to set up port forwarding via ssh: - -{% highlight sh %} -{% raw %} -boot2docker ssh -L8080:localhost:8080 -{% endraw %} -{% endhighlight %} - -List the nodes in your cluster by running: - -{% highlight sh %} -{% raw %} -kubectl get nodes -{% endraw %} -{% endhighlight %} - -This should print: - -{% highlight console %} -{% raw %} -NAME LABELS STATUS -127.0.0.1 Ready -{% endraw %} -{% endhighlight %} - -If you are running different Kubernetes clusters, you may need to specify `-s http://localhost:8080` to select the local cluster. - -### Run an application - -{% highlight sh %} -{% raw %} -kubectl -s http://localhost:8080 run nginx --image=nginx --port=80 -{% endraw %} -{% endhighlight %} - -Now run `docker ps` you should see nginx running. You may need to wait a few minutes for the image to get pulled. - -### Expose it as a service - -{% highlight sh %} -{% raw %} -kubectl expose rc nginx --port=80 -{% endraw %} -{% endhighlight %} - -Run the following command to obtain the IP of this service we just created. There are two IPs, the first one is internal (CLUSTER_IP), and the second one is the external load-balanced IP. - -{% highlight sh %} -{% raw %} -kubectl get svc nginx -{% endraw %} -{% endhighlight %} - -Alternatively, you can obtain only the first IP (CLUSTER_IP) by running: - -{% highlight sh %} -{% raw %} -kubectl get svc nginx --template={{.spec.clusterIP}} -{% endraw %} -{% endhighlight %} - -Hit the webserver with the first IP (CLUSTER_IP): - -{% highlight sh %} -{% raw %} -curl -{% endraw %} -{% endhighlight %} - -Note that you will need run this curl command on your boot2docker VM if you are running on OS X. - -### A note on turning down your cluster - -Many of these containers run under the management of the `kubelet` binary, which attempts to keep containers running, even if they fail. So, in order to turn down -the cluster, you need to first kill the kubelet container, and then any other containers. - +--- +title: "Running Kubernetes locally via Docker" +--- + +## Overview + +The following instructions show you how to set up a simple, single node Kubernetes cluster using Docker. + +Here's a diagram of what the final result will look like: + +![Kubernetes Single Node on Docker](/{{page.version}}/docs/getting-started-guides/k8s-singlenode-docker.png) + + + +{% include pagetoc.html %} + +## Prerequisites + +1. You need to have docker installed on one machine. +2. Your kernel should support memory and swap accounting. Ensure that the +following configs are turned on in your linux kernel: + +{% highlight console %} + CONFIG_RESOURCE_COUNTERS=y + CONFIG_MEMCG=y + CONFIG_MEMCG_SWAP=y + CONFIG_MEMCG_SWAP_ENABLED=y + CONFIG_MEMCG_KMEM=y +{% endhighlight %} + +3. Enable the memory and swap accounting in the kernel, at boot, as command line +parameters as follows: + +{% highlight console %} + GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1" +{% endhighlight %} + + NOTE: The above is specifically for GRUB2. + You can check the command line parameters passed to your kernel by looking at the + output of /proc/cmdline: + +{% highlight console %} + $cat /proc/cmdline + BOOT_IMAGE=/boot/vmlinuz-3.18.4-aufs root=/dev/sda5 ro cgroup_enable=memory + swapaccount=1 +{% endhighlight %} + +### Step One: Run etcd + +{% highlight sh %} +docker run --net=host -d gcr.io/google_containers/etcd:2.0.12 /usr/local/bin/etcd --addr=127.0.0.1:4001 --bind-addr=0.0.0.0:4001 --data-dir=/var/etcd/data +{% endhighlight %} + +### Step Two: Run the master + +{% highlight sh %} +docker run \ + --volume=/:/rootfs:ro \ + --volume=/sys:/sys:ro \ + --volume=/dev:/dev \ + --volume=/var/lib/docker/:/var/lib/docker:ro \ + --volume=/var/lib/kubelet/:/var/lib/kubelet:rw \ + --volume=/var/run:/var/run:rw \ + --net=host \ + --pid=host \ + --privileged=true \ + -d \ + gcr.io/google_containers/hyperkube:v1.0.1 \ + /hyperkube kubelet --containerized --hostname-override="127.0.0.1" --address="0.0.0.0" --api-servers=http://localhost:8080 --config=/etc/kubernetes/manifests +{% endhighlight %} + +This actually runs the kubelet, which in turn runs a [pod](../user-guide/pods) that contains the other master components. + +### Step Three: Run the service proxy + +{% highlight sh %} +docker run -d --net=host --privileged gcr.io/google_containers/hyperkube:v1.0.1 /hyperkube proxy --master=http://127.0.0.1:8080 --v=2 +{% endhighlight %} + +### Test it out + +At this point you should have a running Kubernetes cluster. You can test this by downloading the kubectl +binary +([OS X](https://storage.googleapis.com/kubernetes-release/release/v1.0.1/bin/darwin/amd64/kubectl)) +([linux](https://storage.googleapis.com/kubernetes-release/release/v1.0.1/bin/linux/amd64/kubectl)) + +*Note:* +On OS/X you will need to set up port forwarding via ssh: + +{% highlight sh %} +boot2docker ssh -L8080:localhost:8080 +{% endhighlight %} + +List the nodes in your cluster by running: + +{% highlight sh %} +kubectl get nodes +{% endhighlight %} + +This should print: + +{% highlight console %} +NAME LABELS STATUS +127.0.0.1 Ready +{% endhighlight %} + +If you are running different Kubernetes clusters, you may need to specify `-s http://localhost:8080` to select the local cluster. + +### Run an application + +{% highlight sh %} +kubectl -s http://localhost:8080 run nginx --image=nginx --port=80 +{% endhighlight %} + +Now run `docker ps` you should see nginx running. You may need to wait a few minutes for the image to get pulled. + +### Expose it as a service + +{% highlight sh %} +kubectl expose rc nginx --port=80 +{% endhighlight %} + +Run the following command to obtain the IP of this service we just created. There are two IPs, the first one is internal (CLUSTER_IP), and the second one is the external load-balanced IP. + +{% highlight sh %} +kubectl get svc nginx +{% endhighlight %} + +Alternatively, you can obtain only the first IP (CLUSTER_IP) by running: + +{% highlight sh %} +kubectl get svc nginx --template={{.spec.clusterIP}} +{% endhighlight %} + +Hit the webserver with the first IP (CLUSTER_IP): + +{% highlight sh %} +curl +{% endhighlight %} + +Note that you will need run this curl command on your boot2docker VM if you are running on OS X. + +### A note on turning down your cluster + +Many of these containers run under the management of the `kubelet` binary, which attempts to keep containers running, even if they fail. So, in order to turn down +the cluster, you need to first kill the kubelet container, and then any other containers. + You may use `docker kill $(docker ps -aq)`, note this removes _all_ containers running under Docker, so use with caution. \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/fedora/fedora-calico.md b/v1.1/docs/getting-started-guides/fedora/fedora-calico.md index cfef67275e..751a9ab3bb 100644 --- a/v1.1/docs/getting-started-guides/fedora/fedora-calico.md +++ b/v1.1/docs/getting-started-guides/fedora/fedora-calico.md @@ -1,26 +1,7 @@ --- title: "Running Kubernetes with Calico Networking on a Digital Ocean Fedora Host" --- - - -Running Kubernetes with [Calico Networking](http://projectcalico.org) on a [Digital Ocean](http://digitalocean.com) [Fedora Host](http://fedoraproject.org) ------------------------------------------------------ - -## Table of Contents - -* [Prerequisites](#prerequisites) -* [Overview](#overview) -* [Setup Communication Between Hosts](#setup-communication-between-hosts) -* [Setup Master](#setup-master) - * [Install etcd](#install-etcd) - * [Install Kubernetes](#install-kubernetes) - * [Install Calico](#install-calico) -* [Setup Node](#setup-node) - * [Configure the Virtual Interface - cbr0](#configure-the-virtual-interface---cbr0) - * [Install Docker](#install-docker) - * [Install Calico](#install-calico-1) - * [Install Kubernetes](#install-kubernetes-1) -* [Check Running Cluster](#check-running-cluster) +{% include pagetoc.html %} ## Prerequisites @@ -60,10 +41,10 @@ Ensure you substitute the IP Addresses and Hostnames used in this guide with one Digital Ocean private networking configures a private network on eth1 for each host. To simplify communication between the hosts, we will add an entry to /etc/hosts so that all hosts in the cluster can hostname-resolve one another to this interface. **It is important that the hostname resolves to this interface instead of eth0, as all Kubernetes and Calico services will be running on it.** ``` -{% raw %} + echo "10.134.251.56 kube-master" >> /etc/hosts echo "10.134.251.55 kube-node-1" >> /etc/hosts -{% endraw %} + ``` >Make sure that communication works between kube-master and each kube-node by using a utility such as ping. @@ -75,19 +56,19 @@ echo "10.134.251.55 kube-node-1" >> /etc/hosts * Both Calico and Kubernetes use etcd as their datastore. We will run etcd on Master and point all Kubernetes and Calico services at it. ``` -{% raw %} + yum -y install etcd -{% endraw %} + ``` * Edit `/etc/etcd/etcd.conf` ``` -{% raw %} + ETCD_LISTEN_CLIENT_URLS="http://kube-master:4001" ETCD_ADVERTISE_CLIENT_URLS="http://kube-master:4001" -{% endraw %} + ``` ### Install Kubernetes @@ -95,24 +76,24 @@ ETCD_ADVERTISE_CLIENT_URLS="http://kube-master:4001" * Run the following command on Master to install the latest Kubernetes (as well as docker): ``` -{% raw %} + yum -y install kubernetes -{% endraw %} + ``` * Edit `/etc/kubernetes/config ` ``` -{% raw %} + # How the controller-manager, scheduler, and proxy find the apiserver KUBE_MASTER="--master=http://kube-master:8080" -{% endraw %} + ``` * Edit `/etc/kubernetes/apiserver` ``` -{% raw %} + # The address on the local server to listen to. KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0" @@ -120,29 +101,29 @@ KUBE_ETCD_SERVERS="--etcd-servers=http://kube-master:4001" # Remove ServiceAccount from this line to run without API Tokens KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota" -{% endraw %} + ``` * Create /var/run/kubernetes on master: ``` -{% raw %} + mkdir /var/run/kubernetes chown kube:kube /var/run/kubernetes chmod 750 /var/run/kubernetes -{% endraw %} + ``` * Start the appropriate services on master: ``` -{% raw %} + for SERVICE in etcd kube-apiserver kube-controller-manager kube-scheduler; do systemctl restart $SERVICE systemctl enable $SERVICE systemctl status $SERVICE done -{% endraw %} + ``` ### Install Calico @@ -151,17 +132,17 @@ Next, we'll launch Calico on Master to allow communication between Pods and any * Install calicoctl, the calico configuration tool. ``` -{% raw %} + wget https://github.com/Metaswitch/calico-docker/releases/download/v0.5.5/calicoctl chmod +x ./calicoctl sudo mv ./calicoctl /usr/bin -{% endraw %} + ``` * Create `/etc/systemd/system/calico-node.service` ``` -{% raw %} + [Unit] Description=calicoctl node Requires=docker.service @@ -176,7 +157,7 @@ ExecStart=/usr/bin/calicoctl node --ip=10.134.251.56 --detach=false [Install] WantedBy=multi-user.target -{% endraw %} + ``` >Be sure to substitute `--ip=10.134.251.56` with your Master's eth1 IP Address. @@ -184,10 +165,10 @@ WantedBy=multi-user.target * Start Calico ``` -{% raw %} + systemctl enable calico-node.service systemctl start calico-node.service -{% endraw %} + ``` >Starting calico for the first time may take a few minutes as the calico-node docker image is downloaded. @@ -201,14 +182,14 @@ By default, docker will create and run on a virtual interface called `docker0`. * Add a virtual interface by creating `/etc/sysconfig/network-scripts/ifcfg-cbr0`: ``` -{% raw %} + DEVICE=cbr0 TYPE=Bridge IPADDR=192.168.1.1 NETMASK=255.255.255.0 ONBOOT=yes BOOTPROTO=static -{% endraw %} + ``` >**Note for Multi-Node Clusters:** Each node should be assigned an IP address on a unique subnet. In this example, node-1 is using 192.168.1.1/24, so node-2 should be assigned another pool on the 192.168.x.0/24 subnet, e.g. 192.168.2.1/24. @@ -216,9 +197,9 @@ BOOTPROTO=static * Ensure that your system has bridge-utils installed. Then, restart the networking daemon to activate the new interface ``` -{% raw %} + systemctl restart network.service -{% endraw %} + ``` ### Install Docker @@ -226,25 +207,25 @@ systemctl restart network.service * Install Docker ``` -{% raw %} + yum -y install docker -{% endraw %} + ``` * Configure docker to run on `cbr0` by editing `/etc/sysconfig/docker-network`: ``` -{% raw %} + DOCKER_NETWORK_OPTIONS="--bridge=cbr0 --iptables=false --ip-masq=false" -{% endraw %} + ``` * Start docker ``` -{% raw %} + systemctl start docker -{% endraw %} + ``` ### Install Calico @@ -252,17 +233,17 @@ systemctl start docker * Install calicoctl, the calico configuration tool. ``` -{% raw %} + wget https://github.com/Metaswitch/calico-docker/releases/download/v0.5.5/calicoctl chmod +x ./calicoctl sudo mv ./calicoctl /usr/bin -{% endraw %} + ``` * Create `/etc/systemd/system/calico-node.service` ``` -{% raw %} + [Unit] Description=calicoctl node Requires=docker.service @@ -277,7 +258,7 @@ ExecStart=/usr/bin/calicoctl node --ip=10.134.251.55 --detach=false --kubernetes [Install] WantedBy=multi-user.target -{% endraw %} + ``` > Note: You must replace the IP address with your node's eth1 IP Address! @@ -285,10 +266,10 @@ WantedBy=multi-user.target * Start Calico ``` -{% raw %} + systemctl enable calico-node.service systemctl start calico-node.service -{% endraw %} + ``` * Configure the IP Address Pool @@ -296,9 +277,9 @@ systemctl start calico-node.service Most Kubernetes application deployments will require communication between Pods and the kube-apiserver on Master. On a standard Digital Ocean Private Network, requests sent from Pods to the kube-apiserver will not be returned as the networking fabric will drop response packets destined for any 192.168.0.0/16 address. To resolve this, you can have calicoctl add a masquerade rule to all outgoing traffic on the node: ``` -{% raw %} + ETCD_AUTHORITY=kube-master:4001 calicoctl pool add 192.168.0.0/16 --nat-outgoing -{% endraw %} + ``` ### Install Kubernetes @@ -306,18 +287,18 @@ ETCD_AUTHORITY=kube-master:4001 calicoctl pool add 192.168.0.0/16 --nat-outgoing * First, install Kubernetes. ``` -{% raw %} + yum -y install kubernetes -{% endraw %} + ``` * Edit `/etc/kubernetes/config` ``` -{% raw %} + # How the controller-manager, scheduler, and proxy find the apiserver KUBE_MASTER="--master=http://kube-master:8080" -{% endraw %} + ``` * Edit `/etc/kubernetes/kubelet` @@ -325,7 +306,7 @@ KUBE_MASTER="--master=http://kube-master:8080" We'll pass in an extra parameter - `--network-plugin=calico` to tell the Kubelet to use the Calico networking plugin. Additionally, we'll add two environment variables that will be used by the Calico networking plugin. ``` -{% raw %} + # The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces) KUBELET_ADDRESS="--address=0.0.0.0" @@ -341,19 +322,19 @@ KUBELET_ARGS="--network-plugin=calico" # The following are variables which the kubelet will pass to the calico-networking plugin ETCD_AUTHORITY="kube-master:4001" KUBE_API_ROOT="http://kube-master:8080/api/v1" -{% endraw %} + ``` * Start Kubernetes on the node. ``` -{% raw %} + for SERVICE in kube-proxy kubelet; do systemctl restart $SERVICE systemctl enable $SERVICE systemctl status $SERVICE done -{% endraw %} + ``` ## Check Running Cluster @@ -361,11 +342,11 @@ done The cluster should be running! Check that your nodes are reporting as such: ``` -{% raw %} + kubectl get nodes NAME LABELS STATUS kube-node-1 kubernetes.io/hostname=kube-node-1 Ready -{% endraw %} + ``` diff --git a/v1.1/docs/getting-started-guides/fedora/fedora_ansible_config.md b/v1.1/docs/getting-started-guides/fedora/fedora_ansible_config.md index 67884984e0..55a7968da5 100644 --- a/v1.1/docs/getting-started-guides/fedora/fedora_ansible_config.md +++ b/v1.1/docs/getting-started-guides/fedora/fedora_ansible_config.md @@ -1,270 +1,227 @@ ---- -title: "Configuring Kubernetes on Fedora via Ansible" -section: guides ---- - -Configuring Kubernetes on Fedora via Ansible offers a simple way to quickly create a clustered environment with little effort. - -## Table of Contents - -{% include pagetoc.html %} - -## Prerequisites - -1. Host able to run ansible and able to clone the following repo: [kubernetes](https://github.com/kubernetes/kubernetes.git) -2. A Fedora 21+ host to act as cluster master -3. As many Fedora 21+ hosts as you would like, that act as cluster nodes - -The hosts can be virtual or bare metal. Ansible will take care of the rest of the configuration for you - configuring networking, installing packages, handling the firewall, etc. This example will use one master and two nodes. - -## Architecture of the cluster - -A Kubernetes cluster requires etcd, a master, and n nodes, so we will create a cluster with three hosts, for example: - -{% highlight console %} -{% raw %} - master,etcd = kube-master.example.com - node1 = kube-node-01.example.com - node2 = kube-node-02.example.com -{% endraw %} -{% endhighlight %} - -**Make sure your local machine has** - - - ansible (must be 1.9.0+) - - git - - python-netaddr - -If not - -{% highlight sh %} -{% raw %} -yum install -y ansible git python-netaddr -{% endraw %} -{% endhighlight %} - -**Now clone down the Kubernetes repository** - -{% highlight sh %} -{% raw %} -git clone https://github.com/kubernetes/contrib.git -cd contrib/ansible -{% endraw %} -{% endhighlight %} - -**Tell ansible about each machine and its role in your cluster** - -Get the IP addresses from the master and nodes. Add those to the `~/contrib/ansible/inventory` file on the host running Ansible. - -{% highlight console %} -{% raw %} -[masters] -kube-master.example.com - -[etcd] -kube-master.example.com - -[nodes] -kube-node-01.example.com -kube-node-02.example.com -{% endraw %} -{% endhighlight %} - -## Setting up ansible access to your nodes - -If you already are running on a machine which has passwordless ssh access to the kube-master and kube-node-{01,02} nodes, and 'sudo' privileges, simply set the value of `ansible_ssh_user` in `~/contrib/ansible/group_vars/all.yaml` to the username which you use to ssh to the nodes (i.e. `fedora`), and proceed to the next step... - -*Otherwise* setup ssh on the machines like so (you will need to know the root password to all machines in the cluster). - -edit: ~/contrib/ansible/group_vars/all.yml - -{% highlight yaml %} -{% raw %} -ansible_ssh_user: root -{% endraw %} -{% endhighlight %} - -**Configuring ssh access to the cluster** - -If you already have ssh access to every machine using ssh public keys you may skip to [setting up the cluster](#setting-up-the-cluster) - -Make sure your local machine (root) has an ssh key pair if not - -{% highlight sh %} -{% raw %} -ssh-keygen -{% endraw %} -{% endhighlight %} - -Copy the ssh public key to **all** nodes in the cluster - -{% highlight sh %} -{% raw %} -for node in kube-master.example.com kube-node-01.example.com kube-node-02.example.com; do - ssh-copy-id ${node} -done -{% endraw %} -{% endhighlight %} - -## Setting up the cluster - -Although the default value of variables in `~/contrib/ansible/group_vars/all.yml` should be good enough, if not, change them as needed. - -edit: ~/contrib/ansible/group_vars/all.yml - -**Configure access to kubernetes packages** - -Modify `source_type` as below to access kubernetes packages through the package manager. - -{% highlight yaml %} -{% raw %} -source_type: packageManager -{% endraw %} -{% endhighlight %} - -**Configure the IP addresses used for services** - -Each Kubernetes service gets its own IP address. These are not real IPs. You need only select a range of IPs which are not in use elsewhere in your environment. - -{% highlight yaml %} -{% raw %} -kube_service_addresses: 10.254.0.0/16 -{% endraw %} -{% endhighlight %} - -**Managing flannel** - -Modify `flannel_subnet`, `flannel_prefix` and `flannel_host_prefix` only if defaults are not appropriate for your cluster. - - -**Managing add on services in your cluster** - -Set `cluster_logging` to false or true (default) to disable or enable logging with elasticsearch. - -{% highlight yaml %} -{% raw %} -cluster_logging: true -{% endraw %} -{% endhighlight %} - -Turn `cluster_monitoring` to true (default) or false to enable or disable cluster monitoring with heapster and influxdb. - -{% highlight yaml %} -{% raw %} -cluster_monitoring: true -{% endraw %} -{% endhighlight %} - -Turn `dns_setup` to true (recommended) or false to enable or disable whole DNS configuration. - -{% highlight yaml %} -{% raw %} -dns_setup: true -{% endraw %} -{% endhighlight %} - -**Tell ansible to get to work!** - -This will finally setup your whole Kubernetes cluster for you. - -{% highlight sh %} -{% raw %} -cd ~/contrib/ansible/ - -./setup.sh -{% endraw %} -{% endhighlight %} - -## Testing and using your new cluster - -That's all there is to it. It's really that easy. At this point you should have a functioning Kubernetes cluster. - -**Show kubernetes nodes** - -Run the following on the kube-master: - -{% highlight sh %} -{% raw %} -kubectl get nodes -{% endraw %} -{% endhighlight %} - -**Show services running on masters and nodes** - -{% highlight sh %} -{% raw %} -systemctl | grep -i kube -{% endraw %} -{% endhighlight %} - -**Show firewall rules on the masters and nodes** - -{% highlight sh %} -{% raw %} -iptables -nvL -{% endraw %} -{% endhighlight %} - -**Create /tmp/apache.json on the master with the following contents and deploy pod** - -{% highlight json %} -{% raw %} -{ - "kind": "Pod", - "apiVersion": "v1", - "metadata": { - "name": "fedoraapache", - "labels": { - "name": "fedoraapache" - } - }, - "spec": { - "containers": [ - { - "name": "fedoraapache", - "image": "fedora/apache", - "ports": [ - { - "hostPort": 80, - "containerPort": 80 - } - ] - } - ] - } -} -{% endraw %} -{% endhighlight %} - -{% highlight sh %} -{% raw %} -kubectl create -f /tmp/apache.json -{% endraw %} -{% endhighlight %} - -**Check where the pod was created** - -{% highlight sh %} -{% raw %} -kubectl get pods -{% endraw %} -{% endhighlight %} - -**Check Docker status on nodes** - -{% highlight sh %} -{% raw %} -docker ps -docker images -{% endraw %} -{% endhighlight %} - -**After the pod is 'Running' Check web server access on the node** - -{% highlight sh %} -{% raw %} -curl http://localhost -{% endraw %} -{% endhighlight %} - +--- +title: "Configuring Kubernetes on Fedora via Ansible" +--- + +Configuring Kubernetes on Fedora via Ansible offers a simple way to quickly create a clustered environment with little effort. + + + +{% include pagetoc.html %} + +## Prerequisites + +1. Host able to run ansible and able to clone the following repo: [kubernetes](https://github.com/kubernetes/kubernetes.git) +2. A Fedora 21+ host to act as cluster master +3. As many Fedora 21+ hosts as you would like, that act as cluster nodes + +The hosts can be virtual or bare metal. Ansible will take care of the rest of the configuration for you - configuring networking, installing packages, handling the firewall, etc. This example will use one master and two nodes. + +## Architecture of the cluster + +A Kubernetes cluster requires etcd, a master, and n nodes, so we will create a cluster with three hosts, for example: + +{% highlight console %} + master,etcd = kube-master.example.com + node1 = kube-node-01.example.com + node2 = kube-node-02.example.com +{% endhighlight %} + +**Make sure your local machine has** + + - ansible (must be 1.9.0+) + - git + - python-netaddr + +If not + +{% highlight sh %} +yum install -y ansible git python-netaddr +{% endhighlight %} + +**Now clone down the Kubernetes repository** + +{% highlight sh %} +git clone https://github.com/kubernetes/contrib.git +cd contrib/ansible +{% endhighlight %} + +**Tell ansible about each machine and its role in your cluster** + +Get the IP addresses from the master and nodes. Add those to the `~/contrib/ansible/inventory` file on the host running Ansible. + +{% highlight console %} +[masters] +kube-master.example.com + +[etcd] +kube-master.example.com + +[nodes] +kube-node-01.example.com +kube-node-02.example.com +{% endhighlight %} + +## Setting up ansible access to your nodes + +If you already are running on a machine which has passwordless ssh access to the kube-master and kube-node-{01,02} nodes, and 'sudo' privileges, simply set the value of `ansible_ssh_user` in `~/contrib/ansible/group_vars/all.yaml` to the username which you use to ssh to the nodes (i.e. `fedora`), and proceed to the next step... + +*Otherwise* setup ssh on the machines like so (you will need to know the root password to all machines in the cluster). + +edit: ~/contrib/ansible/group_vars/all.yml + +{% highlight yaml %} +ansible_ssh_user: root +{% endhighlight %} + +**Configuring ssh access to the cluster** + +If you already have ssh access to every machine using ssh public keys you may skip to [setting up the cluster](#setting-up-the-cluster) + +Make sure your local machine (root) has an ssh key pair if not + +{% highlight sh %} +ssh-keygen +{% endhighlight %} + +Copy the ssh public key to **all** nodes in the cluster + +{% highlight sh %} +for node in kube-master.example.com kube-node-01.example.com kube-node-02.example.com; do + ssh-copy-id ${node} +done +{% endhighlight %} + +## Setting up the cluster + +Although the default value of variables in `~/contrib/ansible/group_vars/all.yml` should be good enough, if not, change them as needed. + +edit: ~/contrib/ansible/group_vars/all.yml + +**Configure access to kubernetes packages** + +Modify `source_type` as below to access kubernetes packages through the package manager. + +{% highlight yaml %} +source_type: packageManager +{% endhighlight %} + +**Configure the IP addresses used for services** + +Each Kubernetes service gets its own IP address. These are not real IPs. You need only select a range of IPs which are not in use elsewhere in your environment. + +{% highlight yaml %} +kube_service_addresses: 10.254.0.0/16 +{% endhighlight %} + +**Managing flannel** + +Modify `flannel_subnet`, `flannel_prefix` and `flannel_host_prefix` only if defaults are not appropriate for your cluster. + + +**Managing add on services in your cluster** + +Set `cluster_logging` to false or true (default) to disable or enable logging with elasticsearch. + +{% highlight yaml %} +cluster_logging: true +{% endhighlight %} + +Turn `cluster_monitoring` to true (default) or false to enable or disable cluster monitoring with heapster and influxdb. + +{% highlight yaml %} +cluster_monitoring: true +{% endhighlight %} + +Turn `dns_setup` to true (recommended) or false to enable or disable whole DNS configuration. + +{% highlight yaml %} +dns_setup: true +{% endhighlight %} + +**Tell ansible to get to work!** + +This will finally setup your whole Kubernetes cluster for you. + +{% highlight sh %} +cd ~/contrib/ansible/ + +./setup.sh +{% endhighlight %} + +## Testing and using your new cluster + +That's all there is to it. It's really that easy. At this point you should have a functioning Kubernetes cluster. + +**Show kubernetes nodes** + +Run the following on the kube-master: + +{% highlight sh %} +kubectl get nodes +{% endhighlight %} + +**Show services running on masters and nodes** + +{% highlight sh %} +systemctl | grep -i kube +{% endhighlight %} + +**Show firewall rules on the masters and nodes** + +{% highlight sh %} +iptables -nvL +{% endhighlight %} + +**Create /tmp/apache.json on the master with the following contents and deploy pod** + +{% highlight json %} +{ + "kind": "Pod", + "apiVersion": "v1", + "metadata": { + "name": "fedoraapache", + "labels": { + "name": "fedoraapache" + } + }, + "spec": { + "containers": [ + { + "name": "fedoraapache", + "image": "fedora/apache", + "ports": [ + { + "hostPort": 80, + "containerPort": 80 + } + ] + } + ] + } +} +{% endhighlight %} + +{% highlight sh %} +kubectl create -f /tmp/apache.json +{% endhighlight %} + +**Check where the pod was created** + +{% highlight sh %} +kubectl get pods +{% endhighlight %} + +**Check Docker status on nodes** + +{% highlight sh %} +docker ps +docker images +{% endhighlight %} + +**After the pod is 'Running' Check web server access on the node** + +{% highlight sh %} +curl http://localhost +{% endhighlight %} + That's it ! \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/fedora/fedora_manual_config.md b/v1.1/docs/getting-started-guides/fedora/fedora_manual_config.md index f7f1efe873..8bef9df41b 100644 --- a/v1.1/docs/getting-started-guides/fedora/fedora_manual_config.md +++ b/v1.1/docs/getting-started-guides/fedora/fedora_manual_config.md @@ -1,244 +1,212 @@ ---- -title: "Getting started on Fedora" ---- - -Getting started on [Fedora](http://fedoraproject.org) ------------------------------------------------------ - -**Table of Contents** - -- [Prerequisites](#prerequisites) -- [Instructions](#instructions) - -## Prerequisites - -1. You need 2 or more machines with Fedora installed. - -## Instructions - -This is a getting started guide for Fedora. It is a manual configuration so you understand all the underlying packages / services / ports, etc... - -This guide will only get ONE node (previously minion) working. Multiple nodes require a functional [networking configuration](../../admin/networking.html) done outside of Kubernetes. Although the additional Kubernetes configuration requirements should be obvious. - -The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd and the configuration resides in a central location: /etc/kubernetes. We will break the services up between the hosts. The first host, fed-master, will be the Kubernetes master. This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master will also run _etcd_ (not needed if _etcd_ runs on a different host but this guide assumes that _etcd_ and Kubernetes master run on the same host). The remaining host, fed-node will be the node and run kubelet, proxy and docker. - -**System Information:** - -Hosts: - -``` -{% raw %} -fed-master = 192.168.121.9 -fed-node = 192.168.121.65 -{% endraw %} -``` - -**Prepare the hosts:** - -* Install Kubernetes on all hosts - fed-{master,node}. This will also pull in docker. Also install etcd on fed-master. This guide has been tested with kubernetes-0.18 and beyond. -* The [--enablerepo=updates-testing](https://fedoraproject.org/wiki/QA:Updates_Testing) directive in the yum command below will ensure that the most recent Kubernetes version that is scheduled for pre-release will be installed. This should be a more recent version than the Fedora "stable" release for Kubernetes that you would get without adding the directive. -* If you want the very latest Kubernetes release [you can download and yum install the RPM directly from Fedora Koji](http://koji.fedoraproject.org/koji/packageinfo?packageID=19202) instead of using the yum install command below. - -{% highlight sh %} -{% raw %} -yum -y install --enablerepo=updates-testing kubernetes -{% endraw %} -{% endhighlight %} - -* Install etcd and iptables - -{% highlight sh %} -{% raw %} -yum -y install etcd iptables -{% endraw %} -{% endhighlight %} - -* Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS). Make sure that communication works between fed-master and fed-node by using a utility such as ping. - -{% highlight sh %} -{% raw %} -echo "192.168.121.9 fed-master -192.168.121.65 fed-node" >> /etc/hosts -{% endraw %} -{% endhighlight %} - -* Edit /etc/kubernetes/config which will be the same on all hosts (master and node) to contain: - -{% highlight sh %} -{% raw %} -# Comma separated list of nodes in the etcd cluster -KUBE_MASTER="--master=http://fed-master:8080" - -# logging to stderr means we get it in the systemd journal -KUBE_LOGTOSTDERR="--logtostderr=true" - -# journal message level, 0 is debug -KUBE_LOG_LEVEL="--v=0" - -# Should this cluster be allowed to run privileged docker containers -KUBE_ALLOW_PRIV="--allow-privileged=false" -{% endraw %} -{% endhighlight %} - -* Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers. Please note that iptables-services does not exist on default fedora server install. - -{% highlight sh %} -{% raw %} -systemctl disable iptables-services firewalld -systemctl stop iptables-services firewalld -{% endraw %} -{% endhighlight %} - -**Configure the Kubernetes services on the master.** - -* Edit /etc/kubernetes/apiserver to appear as such. The service-cluster-ip-range IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything. - -{% highlight sh %} -{% raw %} -# The address on the local server to listen to. -KUBE_API_ADDRESS="--address=0.0.0.0" - -# Comma separated list of nodes in the etcd cluster -KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:4001" - -# Address range to use for services -KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" - -# Add your own! -KUBE_API_ARGS="" -{% endraw %} -{% endhighlight %} - -* Edit /etc/etcd/etcd.conf,let the etcd to listen all the ip instead of 127.0.0.1, if not, you will get the error like "connection refused". Note that Fedora 22 uses etcd 2.0, One of the changes in etcd 2.0 is that now uses port 2379 and 2380 (as opposed to etcd 0.46 which userd 4001 and 7001). - -{% highlight sh %} -{% raw %} -ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:4001" -{% endraw %} -{% endhighlight %} - -* Create /var/run/kubernetes on master: - -{% highlight sh %} -{% raw %} -mkdir /var/run/kubernetes -chown kube:kube /var/run/kubernetes -chmod 750 /var/run/kubernetes -{% endraw %} -{% endhighlight %} - -* Start the appropriate services on master: - -{% highlight sh %} -{% raw %} -for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do - systemctl restart $SERVICES - systemctl enable $SERVICES - systemctl status $SERVICES -done -{% endraw %} -{% endhighlight %} - -* Addition of nodes: - -* Create following node.json file on Kubernetes master node: - -{% highlight json %} -{% raw %} -{ - "apiVersion": "v1", - "kind": "Node", - "metadata": { - "name": "fed-node", - "labels":{ "name": "fed-node-label"} - }, - "spec": { - "externalID": "fed-node" - } -} -{% endraw %} -{% endhighlight %} - -Now create a node object internally in your Kubernetes cluster by running: - -{% highlight console %} -{% raw %} -$ kubectl create -f ./node.json - -$ kubectl get nodes -NAME LABELS STATUS -fed-node name=fed-node-label Unknown -{% endraw %} -{% endhighlight %} - -Please note that in the above, it only creates a representation for the node -_fed-node_ internally. It does not provision the actual _fed-node_. Also, it -is assumed that _fed-node_ (as specified in `name`) can be resolved and is -reachable from Kubernetes master node. This guide will discuss how to provision -a Kubernetes node (fed-node) below. - -**Configure the Kubernetes services on the node.** - -***We need to configure the kubelet on the node.*** - -* Edit /etc/kubernetes/kubelet to appear as such: - -{% highlight sh %} -{% raw %} -### -# Kubernetes kubelet (node) config - -# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces) -KUBELET_ADDRESS="--address=0.0.0.0" - -# You may leave this blank to use the actual hostname -KUBELET_HOSTNAME="--hostname-override=fed-node" - -# location of the api-server -KUBELET_API_SERVER="--api-servers=http://fed-master:8080" - -# Add your own! -#KUBELET_ARGS="" -{% endraw %} -{% endhighlight %} - -* Start the appropriate services on the node (fed-node). - -{% highlight sh %} -{% raw %} -for SERVICES in kube-proxy kubelet docker; do - systemctl restart $SERVICES - systemctl enable $SERVICES - systemctl status $SERVICES -done -{% endraw %} -{% endhighlight %} - -* Check to make sure now the cluster can see the fed-node on fed-master, and its status changes to _Ready_. - -{% highlight console %} -{% raw %} -kubectl get nodes -NAME LABELS STATUS -fed-node name=fed-node-label Ready -{% endraw %} -{% endhighlight %} - -* Deletion of nodes: - -To delete _fed-node_ from your Kubernetes cluster, one should run the following on fed-master (Please do not do it, it is just for information): - -{% highlight sh %} -{% raw %} -kubectl delete -f ./node.json -{% endraw %} -{% endhighlight %} - -*You should be finished!* - -**The cluster should be running! Launch a test pod.** - -You should have a functional cluster, check out [101](../../../docs/user-guide/walkthrough/README.html)! - - - +--- +title: "Getting started on Fedora" +--- + +Getting started on [Fedora](http://fedoraproject.org) +----------------------------------------------------- + +{% include pagetoc.html %} + +- [Prerequisites](#prerequisites) +- [Instructions](#instructions) + +## Prerequisites + +1. You need 2 or more machines with Fedora installed. + +## Instructions + +This is a getting started guide for Fedora. It is a manual configuration so you understand all the underlying packages / services / ports, etc... + +This guide will only get ONE node (previously minion) working. Multiple nodes require a functional [networking configuration](../../admin/networking) done outside of Kubernetes. Although the additional Kubernetes configuration requirements should be obvious. + +The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd and the configuration resides in a central location: /etc/kubernetes. We will break the services up between the hosts. The first host, fed-master, will be the Kubernetes master. This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master will also run _etcd_ (not needed if _etcd_ runs on a different host but this guide assumes that _etcd_ and Kubernetes master run on the same host). The remaining host, fed-node will be the node and run kubelet, proxy and docker. + +**System Information:** + +Hosts: + +``` +fed-master = 192.168.121.9 +fed-node = 192.168.121.65 +``` + +**Prepare the hosts:** + +* Install Kubernetes on all hosts - fed-{master,node}. This will also pull in docker. Also install etcd on fed-master. This guide has been tested with kubernetes-0.18 and beyond. +* The [--enablerepo=updates-testing](https://fedoraproject.org/wiki/QA:Updates_Testing) directive in the yum command below will ensure that the most recent Kubernetes version that is scheduled for pre-release will be installed. This should be a more recent version than the Fedora "stable" release for Kubernetes that you would get without adding the directive. +* If you want the very latest Kubernetes release [you can download and yum install the RPM directly from Fedora Koji](http://koji.fedoraproject.org/koji/packageinfo?packageID=19202) instead of using the yum install command below. + +{% highlight sh %} +yum -y install --enablerepo=updates-testing kubernetes +{% endhighlight %} + +* Install etcd and iptables + +{% highlight sh %} +yum -y install etcd iptables +{% endhighlight %} + +* Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS). Make sure that communication works between fed-master and fed-node by using a utility such as ping. + +{% highlight sh %} +echo "192.168.121.9 fed-master +192.168.121.65 fed-node" >> /etc/hosts +{% endhighlight %} + +* Edit /etc/kubernetes/config which will be the same on all hosts (master and node) to contain: + +{% highlight sh %} +# Comma separated list of nodes in the etcd cluster +KUBE_MASTER="--master=http://fed-master:8080" + +# logging to stderr means we get it in the systemd journal +KUBE_LOGTOSTDERR="--logtostderr=true" + +# journal message level, 0 is debug +KUBE_LOG_LEVEL="--v=0" + +# Should this cluster be allowed to run privileged docker containers +KUBE_ALLOW_PRIV="--allow-privileged=false" +{% endhighlight %} + +* Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers. Please note that iptables-services does not exist on default fedora server install. + +{% highlight sh %} +systemctl disable iptables-services firewalld +systemctl stop iptables-services firewalld +{% endhighlight %} + +**Configure the Kubernetes services on the master.** + +* Edit /etc/kubernetes/apiserver to appear as such. The service-cluster-ip-range IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything. + +{% highlight sh %} +# The address on the local server to listen to. +KUBE_API_ADDRESS="--address=0.0.0.0" + +# Comma separated list of nodes in the etcd cluster +KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:4001" + +# Address range to use for services +KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" + +# Add your own! +KUBE_API_ARGS="" +{% endhighlight %} + +* Edit /etc/etcd/etcd.conf,let the etcd to listen all the ip instead of 127.0.0.1, if not, you will get the error like "connection refused". Note that Fedora 22 uses etcd 2.0, One of the changes in etcd 2.0 is that now uses port 2379 and 2380 (as opposed to etcd 0.46 which userd 4001 and 7001). + +{% highlight sh %} +ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:4001" +{% endhighlight %} + +* Create /var/run/kubernetes on master: + +{% highlight sh %} +mkdir /var/run/kubernetes +chown kube:kube /var/run/kubernetes +chmod 750 /var/run/kubernetes +{% endhighlight %} + +* Start the appropriate services on master: + +{% highlight sh %} +for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do + systemctl restart $SERVICES + systemctl enable $SERVICES + systemctl status $SERVICES +done +{% endhighlight %} + +* Addition of nodes: + +* Create following node.json file on Kubernetes master node: + +{% highlight json %} +{ + "apiVersion": "v1", + "kind": "Node", + "metadata": { + "name": "fed-node", + "labels":{ "name": "fed-node-label"} + }, + "spec": { + "externalID": "fed-node" + } +} +{% endhighlight %} + +Now create a node object internally in your Kubernetes cluster by running: + +{% highlight console %} +$ kubectl create -f ./node.json + +$ kubectl get nodes +NAME LABELS STATUS +fed-node name=fed-node-label Unknown +{% endhighlight %} + +Please note that in the above, it only creates a representation for the node +_fed-node_ internally. It does not provision the actual _fed-node_. Also, it +is assumed that _fed-node_ (as specified in `name`) can be resolved and is +reachable from Kubernetes master node. This guide will discuss how to provision +a Kubernetes node (fed-node) below. + +**Configure the Kubernetes services on the node.** + +***We need to configure the kubelet on the node.*** + +* Edit /etc/kubernetes/kubelet to appear as such: + +{% highlight sh %} +### +# Kubernetes kubelet (node) config + +# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces) +KUBELET_ADDRESS="--address=0.0.0.0" + +# You may leave this blank to use the actual hostname +KUBELET_HOSTNAME="--hostname-override=fed-node" + +# location of the api-server +KUBELET_API_SERVER="--api-servers=http://fed-master:8080" + +# Add your own! +#KUBELET_ARGS="" +{% endhighlight %} + +* Start the appropriate services on the node (fed-node). + +{% highlight sh %} +for SERVICES in kube-proxy kubelet docker; do + systemctl restart $SERVICES + systemctl enable $SERVICES + systemctl status $SERVICES +done +{% endhighlight %} + +* Check to make sure now the cluster can see the fed-node on fed-master, and its status changes to _Ready_. + +{% highlight console %} +kubectl get nodes +NAME LABELS STATUS +fed-node name=fed-node-label Ready +{% endhighlight %} + +* Deletion of nodes: + +To delete _fed-node_ from your Kubernetes cluster, one should run the following on fed-master (Please do not do it, it is just for information): + +{% highlight sh %} +kubectl delete -f ./node.json +{% endhighlight %} + +*You should be finished!* + +**The cluster should be running! Launch a test pod.** + +You should have a functional cluster, check out [101](/{{page.version}}/docs/user-guide/walkthrough/README)! + + + diff --git a/v1.1/docs/getting-started-guides/fedora/flannel_multi_node_cluster.md b/v1.1/docs/getting-started-guides/fedora/flannel_multi_node_cluster.md index 83041ab993..9995a89fa5 100644 --- a/v1.1/docs/getting-started-guides/fedora/flannel_multi_node_cluster.md +++ b/v1.1/docs/getting-started-guides/fedora/flannel_multi_node_cluster.md @@ -1,210 +1,178 @@ ---- -title: "Kubernetes multiple nodes cluster with flannel on Fedora" ---- -## Table of Contents - -{% include pagetoc.html %} - -## Introduction - -This document describes how to deploy Kubernetes on multiple hosts to set up a multi-node cluster and networking with flannel. Follow fedora [getting started guide](fedora_manual_config.html) to setup 1 master (fed-master) and 2 or more nodes. Make sure that all nodes have different names (fed-node1, fed-node2 and so on) and labels (fed-node1-label, fed-node2-label, and so on) to avoid any conflict. Also make sure that the Kubernetes master host is running etcd, kube-controller-manager, kube-scheduler, and kube-apiserver services, and the nodes are running docker, kube-proxy and kubelet services. Now install flannel on Kubernetes nodes. flannel on each node configures an overlay network that docker uses. flannel runs on each node to setup a unique class-C container network. - -## Prerequisites - -1. You need 2 or more machines with Fedora installed. - -## Master Setup - -**Perform following commands on the Kubernetes master** - -* Configure flannel by creating a `flannel-config.json` in your current directory on fed-master. flannel provides udp and vxlan among other overlay networking backend options. In this guide, we choose kernel based vxlan backend. The contents of the json are: - -{% highlight json %} -{% raw %} -{ - "Network": "18.16.0.0/16", - "SubnetLen": 24, - "Backend": { - "Type": "vxlan", - "VNI": 1 - } -} -{% endraw %} -{% endhighlight %} - -**NOTE:** Choose an IP range that is *NOT* part of the public IP address range. - -* Add the configuration to the etcd server on fed-master. - -{% highlight sh %} -{% raw %} -etcdctl set /coreos.com/network/config < flannel-config.json -{% endraw %} -{% endhighlight %} - -* Verify the key exists in the etcd server on fed-master. - -{% highlight sh %} -{% raw %} -etcdctl get /coreos.com/network/config -{% endraw %} -{% endhighlight %} - -## Node Setup - -**Perform following commands on all Kubernetes nodes** - -* Edit the flannel configuration file /etc/sysconfig/flanneld as follows: - -{% highlight sh %} -{% raw %} -# Flanneld configuration options - -# etcd url location. Point this to the server where etcd runs -FLANNEL_ETCD="http://fed-master:4001" - -# etcd config key. This is the configuration key that flannel queries -# For address range assignment -FLANNEL_ETCD_KEY="/coreos.com/network" - -# Any additional options that you want to pass -FLANNEL_OPTIONS="" -{% endraw %} -{% endhighlight %} - -**Note:** By default, flannel uses the interface for the default route. If you have multiple interfaces and would like to use an interface other than the default route one, you could add "-iface=" to FLANNEL_OPTIONS. For additional options, run `flanneld --help` on command line. - -* Enable the flannel service. - -{% highlight sh %} -{% raw %} -systemctl enable flanneld -{% endraw %} -{% endhighlight %} - -* If docker is not running, then starting flannel service is enough and skip the next step. - -{% highlight sh %} -{% raw %} -systemctl start flanneld -{% endraw %} -{% endhighlight %} - -* If docker is already running, then stop docker, delete docker bridge (docker0), start flanneld and restart docker as follows. Another alternative is to just reboot the system (`systemctl reboot`). - -{% highlight sh %} -{% raw %} -systemctl stop docker -ip link delete docker0 -systemctl start flanneld -systemctl start docker -{% endraw %} -{% endhighlight %} - -*** - -## **Test the cluster and flannel configuration** - -* Now check the interfaces on the nodes. Notice there is now a flannel.1 interface, and the ip addresses of docker0 and flannel.1 interfaces are in the same network. You will notice that docker0 is assigned a subnet (18.16.29.0/24 as shown below) on each Kubernetes node out of the IP range configured above. A working output should look like this: - -{% highlight console %} -{% raw %} -# ip -4 a|grep inet - inet 127.0.0.1/8 scope host lo - inet 192.168.122.77/24 brd 192.168.122.255 scope global dynamic eth0 - inet 18.16.29.0/16 scope global flannel.1 - inet 18.16.29.1/24 scope global docker0 -{% endraw %} -{% endhighlight %} - -* From any node in the cluster, check the cluster members by issuing a query to etcd server via curl (only partial output is shown using `grep -E "\{|\}|key|value"`). If you set up a 1 master and 3 nodes cluster, you should see one block for each node showing the subnets they have been assigned. You can associate those subnets to each node by the MAC address (VtepMAC) and IP address (Public IP) that is listed in the output. - -{% highlight sh %} -{% raw %} -curl -s http://fed-master:4001/v2/keys/coreos.com/network/subnets | python -mjson.tool -{% endraw %} -{% endhighlight %} - -{% highlight json %} -{% raw %} -{ - "node": { - "key": "/coreos.com/network/subnets", - { - "key": "/coreos.com/network/subnets/18.16.29.0-24", - "value": "{\"PublicIP\":\"192.168.122.77\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"46:f1:d0:18:d0:65\"}}" - }, - { - "key": "/coreos.com/network/subnets/18.16.83.0-24", - "value": "{\"PublicIP\":\"192.168.122.36\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"ca:38:78:fc:72:29\"}}" - }, - { - "key": "/coreos.com/network/subnets/18.16.90.0-24", - "value": "{\"PublicIP\":\"192.168.122.127\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"92:e2:80:ba:2d:4d\"}}" - } - } -} -{% endraw %} -{% endhighlight %} - -* From all nodes, review the `/run/flannel/subnet.env` file. This file was generated automatically by flannel. - -{% highlight console %} -{% raw %} -# cat /run/flannel/subnet.env -FLANNEL_SUBNET=18.16.29.1/24 -FLANNEL_MTU=1450 -FLANNEL_IPMASQ=false -{% endraw %} -{% endhighlight %} - -* At this point, we have etcd running on the Kubernetes master, and flannel / docker running on Kubernetes nodes. Next steps are for testing cross-host container communication which will confirm that docker and flannel are configured properly. - -* Issue the following commands on any 2 nodes: - -{% highlight console %} -{% raw %} -# docker run -it fedora:latest bash -bash-4.3# -{% endraw %} -{% endhighlight %} - -* This will place you inside the container. Install iproute and iputils packages to install ip and ping utilities. Due to a [bug](https://bugzilla.redhat.com/show_bug.cgi?id=1142311), it is required to modify capabilities of ping binary to work around "Operation not permitted" error. - -{% highlight console %} -{% raw %} -bash-4.3# yum -y install iproute iputils -bash-4.3# setcap cap_net_raw-ep /usr/bin/ping -{% endraw %} -{% endhighlight %} - -* Now note the IP address on the first node: - -{% highlight console %} -{% raw %} -bash-4.3# ip -4 a l eth0 | grep inet - inet 18.16.29.4/24 scope global eth0 -{% endraw %} -{% endhighlight %} - -* And also note the IP address on the other node: - -{% highlight console %} -{% raw %} -bash-4.3# ip a l eth0 | grep inet - inet 18.16.90.4/24 scope global eth0 -{% endraw %} -{% endhighlight %} - -* Now ping from the first node to the other node: - -{% highlight console %} -{% raw %} -bash-4.3# ping 18.16.90.4 -PING 18.16.90.4 (18.16.90.4) 56(84) bytes of data. -64 bytes from 18.16.90.4: icmp_seq=1 ttl=62 time=0.275 ms -64 bytes from 18.16.90.4: icmp_seq=2 ttl=62 time=0.372 ms -{% endraw %} -{% endhighlight %} - +--- +title: "Kubernetes multiple nodes cluster with flannel on Fedora" +--- + + +{% include pagetoc.html %} + +## Introduction + +This document describes how to deploy Kubernetes on multiple hosts to set up a multi-node cluster and networking with flannel. Follow fedora [getting started guide](fedora_manual_config) to setup 1 master (fed-master) and 2 or more nodes. Make sure that all nodes have different names (fed-node1, fed-node2 and so on) and labels (fed-node1-label, fed-node2-label, and so on) to avoid any conflict. Also make sure that the Kubernetes master host is running etcd, kube-controller-manager, kube-scheduler, and kube-apiserver services, and the nodes are running docker, kube-proxy and kubelet services. Now install flannel on Kubernetes nodes. flannel on each node configures an overlay network that docker uses. flannel runs on each node to setup a unique class-C container network. + +## Prerequisites + +1. You need 2 or more machines with Fedora installed. + +## Master Setup + +**Perform following commands on the Kubernetes master** + +* Configure flannel by creating a `flannel-config.json` in your current directory on fed-master. flannel provides udp and vxlan among other overlay networking backend options. In this guide, we choose kernel based vxlan backend. The contents of the json are: + +{% highlight json %} +{ + "Network": "18.16.0.0/16", + "SubnetLen": 24, + "Backend": { + "Type": "vxlan", + "VNI": 1 + } +} +{% endhighlight %} + +**NOTE:** Choose an IP range that is *NOT* part of the public IP address range. + +* Add the configuration to the etcd server on fed-master. + +{% highlight sh %} +etcdctl set /coreos.com/network/config < flannel-config.json +{% endhighlight %} + +* Verify the key exists in the etcd server on fed-master. + +{% highlight sh %} +etcdctl get /coreos.com/network/config +{% endhighlight %} + +## Node Setup + +**Perform following commands on all Kubernetes nodes** + +* Edit the flannel configuration file /etc/sysconfig/flanneld as follows: + +{% highlight sh %} +# Flanneld configuration options + +# etcd url location. Point this to the server where etcd runs +FLANNEL_ETCD="http://fed-master:4001" + +# etcd config key. This is the configuration key that flannel queries +# For address range assignment +FLANNEL_ETCD_KEY="/coreos.com/network" + +# Any additional options that you want to pass +FLANNEL_OPTIONS="" +{% endhighlight %} + +**Note:** By default, flannel uses the interface for the default route. If you have multiple interfaces and would like to use an interface other than the default route one, you could add "-iface=" to FLANNEL_OPTIONS. For additional options, run `flanneld --help` on command line. + +* Enable the flannel service. + +{% highlight sh %} +systemctl enable flanneld +{% endhighlight %} + +* If docker is not running, then starting flannel service is enough and skip the next step. + +{% highlight sh %} +systemctl start flanneld +{% endhighlight %} + +* If docker is already running, then stop docker, delete docker bridge (docker0), start flanneld and restart docker as follows. Another alternative is to just reboot the system (`systemctl reboot`). + +{% highlight sh %} +systemctl stop docker +ip link delete docker0 +systemctl start flanneld +systemctl start docker +{% endhighlight %} + +*** + +## **Test the cluster and flannel configuration** + +* Now check the interfaces on the nodes. Notice there is now a flannel.1 interface, and the ip addresses of docker0 and flannel.1 interfaces are in the same network. You will notice that docker0 is assigned a subnet (18.16.29.0/24 as shown below) on each Kubernetes node out of the IP range configured above. A working output should look like this: + +{% highlight console %} +# ip -4 a|grep inet + inet 127.0.0.1/8 scope host lo + inet 192.168.122.77/24 brd 192.168.122.255 scope global dynamic eth0 + inet 18.16.29.0/16 scope global flannel.1 + inet 18.16.29.1/24 scope global docker0 +{% endhighlight %} + +* From any node in the cluster, check the cluster members by issuing a query to etcd server via curl (only partial output is shown using `grep -E "\{|\}|key|value"`). If you set up a 1 master and 3 nodes cluster, you should see one block for each node showing the subnets they have been assigned. You can associate those subnets to each node by the MAC address (VtepMAC) and IP address (Public IP) that is listed in the output. + +{% highlight sh %} +curl -s http://fed-master:4001/v2/keys/coreos.com/network/subnets | python -mjson.tool +{% endhighlight %} + +{% highlight json %} +{ + "node": { + "key": "/coreos.com/network/subnets", + { + "key": "/coreos.com/network/subnets/18.16.29.0-24", + "value": "{\"PublicIP\":\"192.168.122.77\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"46:f1:d0:18:d0:65\"}}" + }, + { + "key": "/coreos.com/network/subnets/18.16.83.0-24", + "value": "{\"PublicIP\":\"192.168.122.36\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"ca:38:78:fc:72:29\"}}" + }, + { + "key": "/coreos.com/network/subnets/18.16.90.0-24", + "value": "{\"PublicIP\":\"192.168.122.127\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"92:e2:80:ba:2d:4d\"}}" + } + } +} +{% endhighlight %} + +* From all nodes, review the `/run/flannel/subnet.env` file. This file was generated automatically by flannel. + +{% highlight console %} +# cat /run/flannel/subnet.env +FLANNEL_SUBNET=18.16.29.1/24 +FLANNEL_MTU=1450 +FLANNEL_IPMASQ=false +{% endhighlight %} + +* At this point, we have etcd running on the Kubernetes master, and flannel / docker running on Kubernetes nodes. Next steps are for testing cross-host container communication which will confirm that docker and flannel are configured properly. + +* Issue the following commands on any 2 nodes: + +{% highlight console %} +# docker run -it fedora:latest bash +bash-4.3# +{% endhighlight %} + +* This will place you inside the container. Install iproute and iputils packages to install ip and ping utilities. Due to a [bug](https://bugzilla.redhat.com/show_bug.cgi?id=1142311), it is required to modify capabilities of ping binary to work around "Operation not permitted" error. + +{% highlight console %} +bash-4.3# yum -y install iproute iputils +bash-4.3# setcap cap_net_raw-ep /usr/bin/ping +{% endhighlight %} + +* Now note the IP address on the first node: + +{% highlight console %} +bash-4.3# ip -4 a l eth0 | grep inet + inet 18.16.29.4/24 scope global eth0 +{% endhighlight %} + +* And also note the IP address on the other node: + +{% highlight console %} +bash-4.3# ip a l eth0 | grep inet + inet 18.16.90.4/24 scope global eth0 +{% endhighlight %} + +* Now ping from the first node to the other node: + +{% highlight console %} +bash-4.3# ping 18.16.90.4 +PING 18.16.90.4 (18.16.90.4) 56(84) bytes of data. +64 bytes from 18.16.90.4: icmp_seq=1 ttl=62 time=0.275 ms +64 bytes from 18.16.90.4: icmp_seq=2 ttl=62 time=0.372 ms +{% endhighlight %} + * Now Kubernetes multi-node cluster is set up with overlay networking set up by flannel. \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/gce.md b/v1.1/docs/getting-started-guides/gce.md index f9f9cd2bef..713c627ebe 100644 --- a/v1.1/docs/getting-started-guides/gce.md +++ b/v1.1/docs/getting-started-guides/gce.md @@ -1,240 +1,217 @@ ---- -title: "Getting started on Google Compute Engine" -section: guides ---- - -The example below creates a Kubernetes cluster with 4 worker node Virtual Machines and a master Virtual Machine (i.e. 5 VMs in your cluster). This cluster is set up and controlled from your workstation (or wherever you find convenient). - -## Table of Contents - -{% include pagetoc.html %} - -### Before you start - -If you want a simplified getting started experience and GUI for managing clusters, please consider trying [Google Container Engine](https://cloud.google.com/container-engine/) (GKE) for hosted cluster installation and management. - -If you want to use custom binaries or pure open source Kubernetes, please continue with the instructions below. - -### Prerequisites - -1. You need a Google Cloud Platform account with billing enabled. Visit the [Google Developers Console](http://cloud.google.com/console) for more details. -1. Install `gcloud` as necessary. `gcloud` can be installed as a part of the [Google Cloud SDK](https://cloud.google.com/sdk/). -1. Then, make sure you have the `gcloud preview` command line component installed. Run `gcloud preview` at the command line - if it asks to install any components, go ahead and install them. If it simply shows help text, you're good to go. This is required as the cluster setup script uses GCE [Instance Groups](https://cloud.google.com/compute/docs/instance-groups/), which are in the gcloud preview namespace. You will also need to **enable [`Compute Engine Instance Group Manager API`](https://developers.google.com/console/help/new/#activatingapis)** in the developers console. -1. Make sure that gcloud is set to use the Google Cloud Platform project you want. You can check the current project using `gcloud config list project` and change it via `gcloud config set project `. -1. Make sure you have credentials for GCloud by running ` gcloud auth login`. -1. Make sure you can start up a GCE VM from the command line. At least make sure you can do the [Create an instance](https://cloud.google.com/compute/docs/instances/#startinstancegcloud) part of the GCE Quickstart. -1. Make sure you can ssh into the VM without interactive prompts. See the [Log in to the instance](https://cloud.google.com/compute/docs/instances/#sshing) part of the GCE Quickstart. - -### Starting a cluster - -You can install a client and start a cluster with either one of these commands (we list both in case only one is installed on your machine): - - -{% highlight bash %} -{% raw %} - curl -sS https://get.k8s.io | bash -{% endraw %} -{% endhighlight %} - -or - -{% highlight bash %} -{% raw %} -wget -q -O - https://get.k8s.io | bash -{% endraw %} -{% endhighlight %} - -Once this command completes, you will have a master VM and four worker VMs, running as a Kubernetes cluster. - -By default, some containers will already be running on your cluster. Containers like `kibana` and `elasticsearch` provide [logging](logging.html), while `heapster` provides [monitoring](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/README.md) services. - -The script run by the commands above creates a cluster with the name/prefix "kubernetes". It defines one specific cluster config, so you can't run it more than once. - -Alternately, you can download and install the latest Kubernetes release from [this page](https://github.com/kubernetes/kubernetes/releases), then run the `/cluster/kube-up.sh` script to start the cluster: - -{% highlight bash %} -{% raw %} -cd kubernetes -cluster/kube-up.sh -{% endraw %} -{% endhighlight %} - -If you want more than one cluster running in your project, want to use a different name, or want a different number of worker nodes, see the `/cluster/gce/config-default.sh` file for more fine-grained configuration before you start up your cluster. - -If you run into trouble, please see the section on [troubleshooting](gce.html#troubleshooting), post to the -[google-containers group](https://groups.google.com/forum/#!forum/google-containers), or come ask questions on [Slack](../troubleshooting.html#slack). - -The next few steps will show you: - -1. how to set up the command line client on your workstation to manage the cluster -1. examples of how to use the cluster -1. how to delete the cluster -1. how to start clusters with non-default options (like larger clusters) - -### Installing the Kubernetes command line tools on your workstation - -The cluster startup script will leave you with a running cluster and a `kubernetes` directory on your workstation. -The next step is to make sure the `kubectl` tool is in your path. - -The [kubectl](../user-guide/kubectl/kubectl.html) tool controls the Kubernetes cluster manager. It lets you inspect your cluster resources, create, delete, and update components, and much more. -You will use it to look at your new cluster and bring up example apps. - -Add the appropriate binary folder to your `PATH` to access kubectl: - -{% highlight bash %} -{% raw %} -# OS X -export PATH=/platforms/darwin/amd64:$PATH - -# Linux -export PATH=/platforms/linux/amd64:$PATH -{% endraw %} -{% endhighlight %} - -**Note**: gcloud also ships with `kubectl`, which by default is added to your path. -However the gcloud bundled kubectl version may be older than the one downloaded by the -get.k8s.io install script. We recommend you use the downloaded binary to avoid -potential issues with client/server version skew. - -#### Enabling bash completion of the Kubernetes command line tools - -You may find it useful to enable `kubectl` bash completion: - -``` -{% raw %} -$ source ./contrib/completions/bash/kubectl -{% endraw %} -``` - -**Note**: This will last for the duration of your bash session. If you want to make this permanent you need to add this line in your bash profile. - -Alternatively, on most linux distributions you can also move the completions file to your bash_completions.d like this: - -``` -{% raw %} -$ cp ./contrib/completions/bash/kubectl /etc/bash_completion.d/ -{% endraw %} -``` - -but then you have to update it when you update kubectl. - -### Getting started with your cluster - -#### Inspect your cluster - -Once `kubectl` is in your path, you can use it to look at your cluster. E.g., running: - -{% highlight console %} -{% raw %} -$ kubectl get --all-namespaces services -{% endraw %} -{% endhighlight %} - -should show a set of [services](../user-guide/services.html) that look something like this: - -{% highlight console %} -{% raw %} -NAMESPACE NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE -default kubernetes 10.0.0.1 443/TCP 1d -kube-system kube-dns 10.0.0.2 53/TCP,53/UDP k8s-app=kube-dns 1d -kube-system kube-ui 10.0.0.3 80/TCP k8s-app=kube-ui 1d -... -{% endraw %} -{% endhighlight %} - -Similarly, you can take a look at the set of [pods](../user-guide/pods.html) that were created during cluster startup. -You can do this via the - -{% highlight console %} -{% raw %} -$ kubectl get --all-namespaces pods -{% endraw %} -{% endhighlight %} - -command. - -You'll see a list of pods that looks something like this (the name specifics will be different): - -{% highlight console %} -{% raw %} -NAMESPACE NAME READY STATUS RESTARTS AGE -kube-system fluentd-cloud-logging-kubernetes-minion-63uo 1/1 Running 0 14m -kube-system fluentd-cloud-logging-kubernetes-minion-c1n9 1/1 Running 0 14m -kube-system fluentd-cloud-logging-kubernetes-minion-c4og 1/1 Running 0 14m -kube-system fluentd-cloud-logging-kubernetes-minion-ngua 1/1 Running 0 14m -kube-system kube-dns-v5-7ztia 3/3 Running 0 15m -kube-system kube-ui-v1-curt1 1/1 Running 0 15m -kube-system monitoring-heapster-v5-ex4u3 1/1 Running 1 15m -kube-system monitoring-influx-grafana-v1-piled 2/2 Running 0 15m -{% endraw %} -{% endhighlight %} - -Some of the pods may take a few seconds to start up (during this time they'll show `Pending`), but check that they all show as `Running` after a short period. - -#### Run some examples - -Then, see [a simple nginx example](../../docs/user-guide/simple-nginx.html) to try out your new cluster. - -For more complete applications, please look in the [examples directory](../../examples/). The [guestbook example](../../examples/guestbook/) is a good "getting started" walkthrough. - -### Tearing down the cluster - -To remove/delete/teardown the cluster, use the `kube-down.sh` script. - -{% highlight bash %} -{% raw %} -cd kubernetes -cluster/kube-down.sh -{% endraw %} -{% endhighlight %} - -Likewise, the `kube-up.sh` in the same directory will bring it back up. You do not need to rerun the `curl` or `wget` command: everything needed to setup the Kubernetes cluster is now on your workstation. - -### Customizing - -The script above relies on Google Storage to stage the Kubernetes release. It -then will start (by default) a single master VM along with 4 worker VMs. You -can tweak some of these parameters by editing `kubernetes/cluster/gce/config-default.sh` -You can view a transcript of a successful cluster creation -[here](https://gist.github.com/satnam6502/fc689d1b46db9772adea). - -### Troubleshooting - -#### Project settings - -You need to have the Google Cloud Storage API, and the Google Cloud Storage -JSON API enabled. It is activated by default for new projects. Otherwise, it -can be done in the Google Cloud Console. See the [Google Cloud Storage JSON -API Overview](https://cloud.google.com/storage/docs/json_api/) for more -details. - -Also ensure that-- as listed in the [Prerequsites section](#prerequisites)-- you've enabled the `Compute Engine Instance Group Manager API`, and can start up a GCE VM from the command line as in the [GCE Quickstart](https://cloud.google.com/compute/docs/quickstart) instructions. - -#### Cluster initialization hang - -If the Kubernetes startup script hangs waiting for the API to be reachable, you can troubleshoot by SSHing into the master and node VMs and looking at logs such as `/var/log/startupscript.log`. - -**Once you fix the issue, you should run `kube-down.sh` to cleanup** after the partial cluster creation, before running `kube-up.sh` to try again. - -#### SSH - -If you're having trouble SSHing into your instances, ensure the GCE firewall -isn't blocking port 22 to your VMs. By default, this should work but if you -have edited firewall rules or created a new non-default network, you'll need to -expose it: `gcloud compute firewall-rules create default-ssh --network= ---description "SSH allowed from anywhere" --allow tcp:22` - -Additionally, your GCE SSH key must either have no passcode or you need to be -using `ssh-agent`. - -#### Networking - -The instances must be able to connect to each other using their private IP. The -script uses the "default" network which should have a firewall rule called -"default-allow-internal" which allows traffic on any port on the private IPs. -If this rule is missing from the default network or if you change the network -being used in `cluster/config-default.sh` create a new rule with the following -field values: - -* Source Ranges: `10.0.0.0/8` +--- +title: "Getting started on Google Compute Engine" +--- + +The example below creates a Kubernetes cluster with 4 worker node Virtual Machines and a master Virtual Machine (i.e. 5 VMs in your cluster). This cluster is set up and controlled from your workstation (or wherever you find convenient). + + + +{% include pagetoc.html %} + +### Before you start + +If you want a simplified getting started experience and GUI for managing clusters, please consider trying [Google Container Engine](https://cloud.google.com/container-engine/) (GKE) for hosted cluster installation and management. + +If you want to use custom binaries or pure open source Kubernetes, please continue with the instructions below. + +### Prerequisites + +1. You need a Google Cloud Platform account with billing enabled. Visit the [Google Developers Console](http://cloud.google.com/console) for more details. +1. Install `gcloud` as necessary. `gcloud` can be installed as a part of the [Google Cloud SDK](https://cloud.google.com/sdk/). +1. Then, make sure you have the `gcloud preview` command line component installed. Run `gcloud preview` at the command line - if it asks to install any components, go ahead and install them. If it simply shows help text, you're good to go. This is required as the cluster setup script uses GCE [Instance Groups](https://cloud.google.com/compute/docs/instance-groups/), which are in the gcloud preview namespace. You will also need to **enable [`Compute Engine Instance Group Manager API`](https://developers.google.com/console/help/new/#activatingapis)** in the developers console. +1. Make sure that gcloud is set to use the Google Cloud Platform project you want. You can check the current project using `gcloud config list project` and change it via `gcloud config set project `. +1. Make sure you have credentials for GCloud by running ` gcloud auth login`. +1. Make sure you can start up a GCE VM from the command line. At least make sure you can do the [Create an instance](https://cloud.google.com/compute/docs/instances/#startinstancegcloud) part of the GCE Quickstart. +1. Make sure you can ssh into the VM without interactive prompts. See the [Log in to the instance](https://cloud.google.com/compute/docs/instances/#sshing) part of the GCE Quickstart. + +### Starting a cluster + +You can install a client and start a cluster with either one of these commands (we list both in case only one is installed on your machine): + + +{% highlight bash %} + curl -sS https://get.k8s.io | bash +{% endhighlight %} + +or + +{% highlight bash %} +wget -q -O - https://get.k8s.io | bash +{% endhighlight %} + +Once this command completes, you will have a master VM and four worker VMs, running as a Kubernetes cluster. + +By default, some containers will already be running on your cluster. Containers like `kibana` and `elasticsearch` provide [logging](logging), while `heapster` provides [monitoring](http://releases.k8s.io/release-1.1/cluster/addons/cluster-monitoring/README.md) services. + +The script run by the commands above creates a cluster with the name/prefix "kubernetes". It defines one specific cluster config, so you can't run it more than once. + +Alternately, you can download and install the latest Kubernetes release from [this page](https://github.com/kubernetes/kubernetes/releases), then run the `/cluster/kube-up.sh` script to start the cluster: + +{% highlight bash %} +cd kubernetes +cluster/kube-up.sh +{% endhighlight %} + +If you want more than one cluster running in your project, want to use a different name, or want a different number of worker nodes, see the `/cluster/gce/config-default.sh` file for more fine-grained configuration before you start up your cluster. + +If you run into trouble, please see the section on [troubleshooting](gce.html#troubleshooting), post to the +[google-containers group](https://groups.google.com/forum/#!forum/google-containers), or come ask questions on [Slack](../troubleshooting.html#slack). + +The next few steps will show you: + +1. how to set up the command line client on your workstation to manage the cluster +1. examples of how to use the cluster +1. how to delete the cluster +1. how to start clusters with non-default options (like larger clusters) + +### Installing the Kubernetes command line tools on your workstation + +The cluster startup script will leave you with a running cluster and a `kubernetes` directory on your workstation. +The next step is to make sure the `kubectl` tool is in your path. + +The [kubectl](../user-guide/kubectl/kubectl) tool controls the Kubernetes cluster manager. It lets you inspect your cluster resources, create, delete, and update components, and much more. +You will use it to look at your new cluster and bring up example apps. + +Add the appropriate binary folder to your `PATH` to access kubectl: + +{% highlight bash %} +# OS X +export PATH=/platforms/darwin/amd64:$PATH + +# Linux +export PATH=/platforms/linux/amd64:$PATH +{% endhighlight %} + +**Note**: gcloud also ships with `kubectl`, which by default is added to your path. +However the gcloud bundled kubectl version may be older than the one downloaded by the +get.k8s.io install script. We recommend you use the downloaded binary to avoid +potential issues with client/server version skew. + +#### Enabling bash completion of the Kubernetes command line tools + +You may find it useful to enable `kubectl` bash completion: + +``` +$ source ./contrib/completions/bash/kubectl +``` + +**Note**: This will last for the duration of your bash session. If you want to make this permanent you need to add this line in your bash profile. + +Alternatively, on most linux distributions you can also move the completions file to your bash_completions.d like this: + +``` +$ cp ./contrib/completions/bash/kubectl /etc/bash_completion.d/ +``` + +but then you have to update it when you update kubectl. + +### Getting started with your cluster + +#### Inspect your cluster + +Once `kubectl` is in your path, you can use it to look at your cluster. E.g., running: + +{% highlight console %} +$ kubectl get --all-namespaces services +{% endhighlight %} + +should show a set of [services](../user-guide/services) that look something like this: + +{% highlight console %} +NAMESPACE NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE +default kubernetes 10.0.0.1 443/TCP 1d +kube-system kube-dns 10.0.0.2 53/TCP,53/UDP k8s-app=kube-dns 1d +kube-system kube-ui 10.0.0.3 80/TCP k8s-app=kube-ui 1d +... +{% endhighlight %} + +Similarly, you can take a look at the set of [pods](../user-guide/pods) that were created during cluster startup. +You can do this via the + +{% highlight console %} +$ kubectl get --all-namespaces pods +{% endhighlight %} + +command. + +You'll see a list of pods that looks something like this (the name specifics will be different): + +{% highlight console %} +NAMESPACE NAME READY STATUS RESTARTS AGE +kube-system fluentd-cloud-logging-kubernetes-minion-63uo 1/1 Running 0 14m +kube-system fluentd-cloud-logging-kubernetes-minion-c1n9 1/1 Running 0 14m +kube-system fluentd-cloud-logging-kubernetes-minion-c4og 1/1 Running 0 14m +kube-system fluentd-cloud-logging-kubernetes-minion-ngua 1/1 Running 0 14m +kube-system kube-dns-v5-7ztia 3/3 Running 0 15m +kube-system kube-ui-v1-curt1 1/1 Running 0 15m +kube-system monitoring-heapster-v5-ex4u3 1/1 Running 1 15m +kube-system monitoring-influx-grafana-v1-piled 2/2 Running 0 15m +{% endhighlight %} + +Some of the pods may take a few seconds to start up (during this time they'll show `Pending`), but check that they all show as `Running` after a short period. + +#### Run some examples + +Then, see [a simple nginx example](/{{page.version}}/docs/user-guide/simple-nginx) to try out your new cluster. + +For more complete applications, please look in the [examples directory](../../examples/). The [guestbook example](../../examples/guestbook/) is a good "getting started" walkthrough. + +### Tearing down the cluster + +To remove/delete/teardown the cluster, use the `kube-down.sh` script. + +{% highlight bash %} +cd kubernetes +cluster/kube-down.sh +{% endhighlight %} + +Likewise, the `kube-up.sh` in the same directory will bring it back up. You do not need to rerun the `curl` or `wget` command: everything needed to setup the Kubernetes cluster is now on your workstation. + +### Customizing + +The script above relies on Google Storage to stage the Kubernetes release. It +then will start (by default) a single master VM along with 4 worker VMs. You +can tweak some of these parameters by editing `kubernetes/cluster/gce/config-default.sh` +You can view a transcript of a successful cluster creation +[here](https://gist.github.com/satnam6502/fc689d1b46db9772adea). + +### Troubleshooting + +#### Project settings + +You need to have the Google Cloud Storage API, and the Google Cloud Storage +JSON API enabled. It is activated by default for new projects. Otherwise, it +can be done in the Google Cloud Console. See the [Google Cloud Storage JSON +API Overview](https://cloud.google.com/storage/docs/json_api/) for more +details. + +Also ensure that-- as listed in the [Prerequsites section](#prerequisites)-- you've enabled the `Compute Engine Instance Group Manager API`, and can start up a GCE VM from the command line as in the [GCE Quickstart](https://cloud.google.com/compute/docs/quickstart) instructions. + +#### Cluster initialization hang + +If the Kubernetes startup script hangs waiting for the API to be reachable, you can troubleshoot by SSHing into the master and node VMs and looking at logs such as `/var/log/startupscript.log`. + +**Once you fix the issue, you should run `kube-down.sh` to cleanup** after the partial cluster creation, before running `kube-up.sh` to try again. + +#### SSH + +If you're having trouble SSHing into your instances, ensure the GCE firewall +isn't blocking port 22 to your VMs. By default, this should work but if you +have edited firewall rules or created a new non-default network, you'll need to +expose it: `gcloud compute firewall-rules create default-ssh --network= +--description "SSH allowed from anywhere" --allow tcp:22` + +Additionally, your GCE SSH key must either have no passcode or you need to be +using `ssh-agent`. + +#### Networking + +The instances must be able to connect to each other using their private IP. The +script uses the "default" network which should have a firewall rule called +"default-allow-internal" which allows traffic on any port on the private IPs. +If this rule is missing from the default network or if you change the network +being used in `cluster/config-default.sh` create a new rule with the following +field values: + +* Source Ranges: `10.0.0.0/8` * Allowed Protocols and Port: `tcp:1-65535;udp:1-65535;icmp` \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/index.md b/v1.1/docs/getting-started-guides/index.md index ab363bf628..53b70a74e2 100644 --- a/v1.1/docs/getting-started-guides/index.md +++ b/v1.1/docs/getting-started-guides/index.md @@ -1,35 +1,16 @@ --- title: "Creating a Kubernetes Cluster" --- - - -Creating a Kubernetes Cluster ----------------------------------------- - Kubernetes can run on a range of platforms, from your laptop, to VMs on a cloud provider, to rack of bare metal servers. The effort required to set up a cluster varies from running a single command to crafting your own customized cluster. We'll guide you in picking a solution that fits for your needs. -**Table of Contents** - - - - [Picking the Right Solution](#picking-the-right-solution) - - [Local-machine Solutions](#local-machine-solutions) - - [Hosted Solutions](#hosted-solutions) - - [Turn-key Cloud Solutions](#turn-key-cloud-solutions) - - [Custom Solutions](#custom-solutions) - - [Cloud](#cloud) - - [On-Premises VMs](#on-premises-vms) - - [Bare Metal](#bare-metal) - - [Integrations](#integrations) - - [Table of Solutions](#table-of-solutions) - - +{% include pagetoc.html %} ## Picking the Right Solution -If you just want to "kick the tires" on Kubernetes, we recommend the [local Docker-based](docker.html) solution. +If you just want to "kick the tires" on Kubernetes, we recommend the [local Docker-based](docker) solution. The local Docker-based solution is one of several [Local cluster](#local-machine-solutions) solutions that are quick to set up, but are limited to running on one machine. @@ -52,9 +33,9 @@ But their size and availability is limited to that of a single machine. The local-machine solutions are: -- [Local Docker-based](docker.html) (recommended starting point) -- [Vagrant](vagrant.html) (works on any platform with Vagrant: Linux, MacOS, or Windows.) -- [No-VM local cluster](locally.html) (Linux only) +- [Local Docker-based](docker) (recommended starting point) +- [Vagrant](vagrant) (works on any platform with Vagrant: Linux, MacOS, or Windows.) +- [No-VM local cluster](locally) (Linux only) ### Hosted Solutions @@ -67,9 +48,9 @@ clusters. These solutions allow you to create Kubernetes clusters on a range of Cloud IaaS providers with only a few commands, and have active community support. -- [GCE](gce.html) -- [AWS](aws.html) -- [Azure](coreos/azure/README.html) +- [GCE](gce) +- [AWS](aws) +- [Azure](/{{page.version}}/docs/getting-started-guides/coreos/azure/README) ### Custom Solutions @@ -79,50 +60,50 @@ base operating systems. If you can find a guide below that matches your needs, use it. It may be a little out of date, but it will be easier than starting from scratch. If you do want to start from scratch because you have special requirements or just because you want to understand what is underneath a Kubernetes -cluster, try the [Getting Started from Scratch](scratch.html) guide. +cluster, try the [Getting Started from Scratch](scratch) guide. If you are interested in supporting Kubernetes on a new platform, check out our [advice for -writing a new solution](../../docs/devel/writing-a-getting-started-guide.html). +writing a new solution](/{{page.version}}/docs/devel/writing-a-getting-started-guide). #### Cloud These solutions are combinations of cloud provider and OS not covered by the above solutions. -- [AWS + coreos](coreos.html) -- [GCE + CoreOS](coreos.html) -- [AWS + Ubuntu](juju.html) -- [Joyent + Ubuntu](juju.html) -- [Rackspace + CoreOS](rackspace.html) +- [AWS + coreos](coreos) +- [GCE + CoreOS](coreos) +- [AWS + Ubuntu](juju) +- [Joyent + Ubuntu](juju) +- [Rackspace + CoreOS](rackspace) #### On-Premises VMs -- [Vagrant](coreos.html) (uses CoreOS and flannel) -- [CloudStack](cloudstack.html) (uses Ansible, CoreOS and flannel) -- [Vmware](vsphere.html) (uses Debian) -- [juju.md](juju.html) (uses Juju, Ubuntu and flannel) -- [Vmware](coreos.html) (uses CoreOS and flannel) -- [libvirt-coreos.md](libvirt-coreos.html) (uses CoreOS) -- [oVirt](ovirt.html) -- [libvirt](fedora/flannel_multi_node_cluster.html) (uses Fedora and flannel) -- [KVM](fedora/flannel_multi_node_cluster.html) (uses Fedora and flannel) +- [Vagrant](coreos) (uses CoreOS and flannel) +- [CloudStack](cloudstack) (uses Ansible, CoreOS and flannel) +- [Vmware](vsphere) (uses Debian) +- [juju.md](juju) (uses Juju, Ubuntu and flannel) +- [Vmware](coreos) (uses CoreOS and flannel) +- [libvirt-coreos.md](libvirt-coreos) (uses CoreOS) +- [oVirt](ovirt) +- [libvirt](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) (uses Fedora and flannel) +- [KVM](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) (uses Fedora and flannel) #### Bare Metal -- [Offline](coreos/bare_metal_offline.html) (no internet required. Uses CoreOS and Flannel) -- [fedora/fedora_ansible_config.md](fedora/fedora_ansible_config.html) -- [Fedora single node](fedora/fedora_manual_config.html) -- [Fedora multi node](fedora/flannel_multi_node_cluster.html) -- [Centos](centos/centos_manual_config.html) -- [Ubuntu](ubuntu.html) -- [Docker Multi Node](docker-multinode.html) +- [Offline](/{{page.version}}/docs/getting-started-guides/coreos/bare_metal_offline) (no internet required. Uses CoreOS and Flannel) +- [fedora/fedora_ansible_config.md](/{{page.version}}/docs/getting-started-guides/fedora/fedora_ansible_config) +- [Fedora single node](/{{page.version}}/docs/getting-started-guides/fedora/fedora_manual_config) +- [Fedora multi node](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) +- [Centos](/{{page.version}}/docs/getting-started-guides/centos/centos_manual_config) +- [Ubuntu](ubuntu) +- [Docker Multi Node](docker-multinode) #### Integrations These solutions provide integration with 3rd party schedulers, resource managers, and/or lower level platforms. -- [Kubernetes on Mesos](mesos.html) +- [Kubernetes on Mesos](mesos) - Instructions specify GCE, but are generic enough to be adapted to most existing Mesos clusters -- [Kubernetes on DCOS](dcos.html) +- [Kubernetes on DCOS](dcos) - Community Edition DCOS uses AWS - Enterprise Edition DCOS supports cloud hosting, on-premise VMs, and bare metal @@ -132,40 +113,40 @@ Here are all the solutions mentioned above in table form. IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level -------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -GKE | | | GCE | [docs](https://cloud.google.com/container-engine) | [✓][3] | Commercial -Vagrant | Saltstack | Fedora | flannel | [docs](vagrant.html) | [✓][2] | Project -GCE | Saltstack | Debian | GCE | [docs](gce.html) | [✓][1] | Project -Azure | CoreOS | CoreOS | Weave | [docs](coreos/azure/README.html) | | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) -Docker Single Node | custom | N/A | local | [docs](docker.html) | | Project ([@brendandburns](https://github.com/brendandburns)) -Docker Multi Node | Flannel | N/A | local | [docs](docker-multinode.html) | | Project ([@brendandburns](https://github.com/brendandburns)) -Bare-metal | Ansible | Fedora | flannel | [docs](fedora/fedora_ansible_config.html) | | Project -Digital Ocean | custom | Fedora | Calico | [docs](fedora/fedora-calico.html) | | Community (@djosborne) -Bare-metal | custom | Fedora | _none_ | [docs](fedora/fedora_manual_config.html) | | Project -Bare-metal | custom | Fedora | flannel | [docs](fedora/flannel_multi_node_cluster.html) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) -libvirt | custom | Fedora | flannel | [docs](fedora/flannel_multi_node_cluster.html) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) -KVM | custom | Fedora | flannel | [docs](fedora/flannel_multi_node_cluster.html) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) -Mesos/Docker | custom | Ubuntu | Docker | [docs](mesos-docker.html) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) -Mesos/GCE | | | | [docs](mesos.html) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) -DCOS | Marathon | CoreOS/Alpine | custom | [docs](dcos.html) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) -AWS | CoreOS | CoreOS | flannel | [docs](coreos.html) | | Community -GCE | CoreOS | CoreOS | flannel | [docs](coreos.html) | | Community ([@pires](https://github.com/pires)) -Vagrant | CoreOS | CoreOS | flannel | [docs](coreos.html) | | Community ([@pires](https://github.com/pires), [@AntonioMeireles](https://github.com/AntonioMeireles)) -Bare-metal (Offline) | CoreOS | CoreOS | flannel | [docs](coreos/bare_metal_offline.html) | | Community ([@jeffbean](https://github.com/jeffbean)) -Bare-metal | CoreOS | CoreOS | Calico | [docs](coreos/bare_metal_calico.html) | | Community ([@caseydavenport](https://github.com/caseydavenport)) -CloudStack | Ansible | CoreOS | flannel | [docs](cloudstack.html) | | Community ([@runseb](https://github.com/runseb)) -Vmware | | Debian | OVS | [docs](vsphere.html) | | Community ([@pietern](https://github.com/pietern)) -Bare-metal | custom | CentOS | _none_ | [docs](centos/centos_manual_config.html) | | Community ([@coolsvap](https://github.com/coolsvap)) -AWS | Juju | Ubuntu | flannel | [docs](juju.html) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -OpenStack/HPCloud | Juju | Ubuntu | flannel | [docs](juju.html) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -Joyent | Juju | Ubuntu | flannel | [docs](juju.html) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -AWS | Saltstack | Ubuntu | OVS | [docs](aws.html) | | Community ([@justinsb](https://github.com/justinsb)) -Bare-metal | custom | Ubuntu | Calico | [docs](ubuntu-calico.html) | | Community ([@djosborne](https://github.com/djosborne)) -Bare-metal | custom | Ubuntu | flannel | [docs](ubuntu.html) | | Community ([@resouer](https://github.com/resouer), [@WIZARD-CXY](https://github.com/WIZARD-CXY)) -Local | | | _none_ | [docs](locally.html) | | Community ([@preillyme](https://github.com/preillyme)) -libvirt/KVM | CoreOS | CoreOS | libvirt/KVM | [docs](libvirt-coreos.html) | | Community ([@lhuard1A](https://github.com/lhuard1A)) -oVirt | | | | [docs](ovirt.html) | | Community ([@simon3z](https://github.com/simon3z)) -Rackspace | CoreOS | CoreOS | flannel | [docs](rackspace.html) | | Community ([@doublerr](https://github.com/doublerr)) -any | any | any | any | [docs](scratch.html) | | Community ([@erictune](https://github.com/erictune)) +GKE | | | GCE | [docs](https://cloud.google.com/container-engine) | ['œ“][3] | Commercial +Vagrant | Saltstack | Fedora | flannel | [docs](vagrant) | ['œ“][2] | Project +GCE | Saltstack | Debian | GCE | [docs](gce) | ['œ“][1] | Project +Azure | CoreOS | CoreOS | Weave | [docs](/{{page.version}}/docs/getting-started-guides/coreos/azure/README) | | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) +Docker Single Node | custom | N/A | local | [docs](docker) | | Project ([@brendandburns](https://github.com/brendandburns)) +Docker Multi Node | Flannel | N/A | local | [docs](docker-multinode) | | Project ([@brendandburns](https://github.com/brendandburns)) +Bare-metal | Ansible | Fedora | flannel | [docs](/{{page.version}}/docs/getting-started-guides/fedora/fedora_ansible_config) | | Project +Digital Ocean | custom | Fedora | Calico | [docs](/{{page.version}}/docs/getting-started-guides/fedora/fedora-calico) | | Community (@djosborne) +Bare-metal | custom | Fedora | _none_ | [docs](/{{page.version}}/docs/getting-started-guides/fedora/fedora_manual_config) | | Project +Bare-metal | custom | Fedora | flannel | [docs](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) +libvirt | custom | Fedora | flannel | [docs](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) +KVM | custom | Fedora | flannel | [docs](/{{page.version}}/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal)) +Mesos/Docker | custom | Ubuntu | Docker | [docs](mesos-docker) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) +Mesos/GCE | | | | [docs](mesos) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) +DCOS | Marathon | CoreOS/Alpine | custom | [docs](dcos) | | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) +AWS | CoreOS | CoreOS | flannel | [docs](coreos) | | Community +GCE | CoreOS | CoreOS | flannel | [docs](coreos) | | Community ([@pires](https://github.com/pires)) +Vagrant | CoreOS | CoreOS | flannel | [docs](coreos) | | Community ([@pires](https://github.com/pires), [@AntonioMeireles](https://github.com/AntonioMeireles)) +Bare-metal (Offline) | CoreOS | CoreOS | flannel | [docs](/{{page.version}}/docs/getting-started-guides/coreos/bare_metal_offline) | | Community ([@jeffbean](https://github.com/jeffbean)) +Bare-metal | CoreOS | CoreOS | Calico | [docs](/{{page.version}}/docs/getting-started-guides/coreos/bare_metal_calico) | | Community ([@caseydavenport](https://github.com/caseydavenport)) +CloudStack | Ansible | CoreOS | flannel | [docs](cloudstack) | | Community ([@runseb](https://github.com/runseb)) +Vmware | | Debian | OVS | [docs](vsphere) | | Community ([@pietern](https://github.com/pietern)) +Bare-metal | custom | CentOS | _none_ | [docs](/{{page.version}}/docs/getting-started-guides/centos/centos_manual_config) | | Community ([@coolsvap](https://github.com/coolsvap)) +AWS | Juju | Ubuntu | flannel | [docs](juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) +OpenStack/HPCloud | Juju | Ubuntu | flannel | [docs](juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) +Joyent | Juju | Ubuntu | flannel | [docs](juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) +AWS | Saltstack | Ubuntu | OVS | [docs](aws) | | Community ([@justinsb](https://github.com/justinsb)) +Bare-metal | custom | Ubuntu | Calico | [docs](ubuntu-calico) | | Community ([@djosborne](https://github.com/djosborne)) +Bare-metal | custom | Ubuntu | flannel | [docs](ubuntu) | | Community ([@resouer](https://github.com/resouer), [@WIZARD-CXY](https://github.com/WIZARD-CXY)) +Local | | | _none_ | [docs](locally) | | Community ([@preillyme](https://github.com/preillyme)) +libvirt/KVM | CoreOS | CoreOS | libvirt/KVM | [docs](libvirt-coreos) | | Community ([@lhuard1A](https://github.com/lhuard1A)) +oVirt | | | | [docs](ovirt) | | Community ([@simon3z](https://github.com/simon3z)) +Rackspace | CoreOS | CoreOS | flannel | [docs](rackspace) | | Community ([@doublerr](https://github.com/doublerr)) +any | any | any | any | [docs](scratch) | | Community ([@erictune](https://github.com/erictune)) *Note*: The above table is ordered by version test/used in notes followed by support level. @@ -176,7 +157,7 @@ Definition of columns: - **OS** is the base operating system of the nodes. - **Config. Mgmt** is the configuration management system that helps install and maintain Kubernetes software on the nodes. -- **Networking** is what implements the [networking model](../../docs/admin/networking.html). Those with networking type +- **Networking** is what implements the [networking model](/{{page.version}}/docs/admin/networking). Those with networking type _none_ may not support more than one node, or may support multiple VM nodes only in the same physical node. - **Conformance** indicates whether a cluster created with this configuration has passed the project's conformance tests for supporting the API and base features of Kubernetes v1.0.0. @@ -195,7 +176,4 @@ Definition of columns: [2]: https://gist.github.com/derekwaynecarr/505e56036cdf010bf6b6 -[3]: https://gist.github.com/erictune/2f39b22f72565365e59b - - - +[3]: https://gist.github.com/erictune/2f39b22f72565365e59b \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/juju.md b/v1.1/docs/getting-started-guides/juju.md index 610b1481bd..4715bd7d3e 100644 --- a/v1.1/docs/getting-started-guides/juju.md +++ b/v1.1/docs/getting-started-guides/juju.md @@ -1,226 +1,223 @@ ---- -title: "Getting started with Juju" -section: guides ---- -[Juju](https://jujucharms.com/docs/stable/about-juju) makes it easy to deploy -Kubernetes by provisioning, installing and configuring all the systems in -the cluster. Once deployed the cluster can easily scale up with one command -to increase the cluster size. - -## Table of Contents - -{% include pagetoc.html %} - -## Prerequisites - -> Note: If you're running kube-up, on Ubuntu - all of the dependencies -> will be handled for you. You may safely skip to the section: -> [Launch Kubernetes Cluster](#launch-kubernetes-cluster) - -### On Ubuntu - -[Install the Juju client](https://jujucharms.com/get-started) on your -local Ubuntu system: - - sudo add-apt-repository ppa:juju/stable - sudo apt-get update - sudo apt-get install juju-core juju-quickstart - - -### With Docker - -If you are not using Ubuntu or prefer the isolation of Docker, you may -run the following: - - mkdir ~/.juju - sudo docker run -v ~/.juju:/home/ubuntu/.juju -ti jujusolutions/jujubox:latest - -At this point from either path you will have access to the `juju -quickstart` command. - -To set up the credentials for your chosen cloud run: - - juju quickstart --constraints="mem=3.75G" -i - -> The `constraints` flag is optional, it changes the size of virtual machines -> that Juju will generate when it requests a new machine. Larger machines -> will run faster but cost more money than smaller machines. - -Follow the dialogue and choose `save` and `use`. Quickstart will now -bootstrap the juju root node and setup the juju web based user -interface. - - -## Launch Kubernetes cluster - -You will need to export the `KUBERNETES_PROVIDER` environment variable before -bringing up the cluster. - - export KUBERNETES_PROVIDER=juju - cluster/kube-up.sh - -If this is your first time running the `kube-up.sh` script, it will install -the required dependencies to get started with Juju, additionally it will -launch a curses based configuration utility allowing you to select your cloud -provider and enter the proper access credentials. - -Next it will deploy the kubernetes master, etcd, 2 nodes with flannel based -Software Defined Networking (SDN) so containers on different hosts can -communicate with each other. - - -## Exploring the cluster - -The `juju status` command provides information about each unit in the cluster: - - $ juju status --format=oneline - - docker/0: 52.4.92.78 (started) - - flannel-docker/0: 52.4.92.78 (started) - - kubernetes/0: 52.4.92.78 (started) - - docker/1: 52.6.104.142 (started) - - flannel-docker/1: 52.6.104.142 (started) - - kubernetes/1: 52.6.104.142 (started) - - etcd/0: 52.5.216.210 (started) 4001/tcp - - juju-gui/0: 52.5.205.174 (started) 80/tcp, 443/tcp - - kubernetes-master/0: 52.6.19.238 (started) 8080/tcp - -You can use `juju ssh` to access any of the units: - - juju ssh kubernetes-master/0 - - -## Run some containers! - -`kubectl` is available on the Kubernetes master node. We'll ssh in to -launch some containers, but one could use `kubectl` locally by setting -`KUBERNETES_MASTER` to point at the ip address of "kubernetes-master/0". - -No pods will be available before starting a container: - - kubectl get pods - NAME READY STATUS RESTARTS AGE - - kubectl get replicationcontrollers - CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS - -We'll follow the aws-coreos example. Create a pod manifest: `pod.json` - -{% highlight json %} -{% raw %} -{ - "apiVersion": "v1", - "kind": "Pod", - "metadata": { - "name": "hello", - "labels": { - "name": "hello", - "environment": "testing" - } - }, - "spec": { - "containers": [{ - "name": "hello", - "image": "quay.io/kelseyhightower/hello", - "ports": [{ - "containerPort": 80, - "hostPort": 80 - }] - }] - } -} -{% endraw %} -{% endhighlight %} - -Create the pod with kubectl: - - kubectl create -f pod.json - - -Get info on the pod: - - kubectl get pods - - -To test the hello app, we need to locate which node is hosting -the container. Better tooling for using Juju to introspect container -is in the works but we can use `juju run` and `juju status` to find -our hello app. - -Exit out of our ssh session and run: - - juju run --unit kubernetes/0 "docker ps -n=1" - ... - juju run --unit kubernetes/1 "docker ps -n=1" - CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - 02beb61339d8 quay.io/kelseyhightower/hello:latest /hello About an hour ago Up About an hour k8s_hello.... - - -We see "kubernetes/1" has our container, we can open port 80: - - juju run --unit kubernetes/1 "open-port 80" - juju expose kubernetes - sudo apt-get install curl - curl $(juju status --format=oneline kubernetes/1 | cut -d' ' -f3) - -Finally delete the pod: - - juju ssh kubernetes-master/0 - kubectl delete pods hello - - -## Scale out cluster - -We can add node units like so: - - juju add-unit docker # creates unit docker/2, kubernetes/2, docker-flannel/2 - -## Launch the "k8petstore" example app - -The [k8petstore example](../../examples/k8petstore/) is available as a -[juju action](https://jujucharms.com/docs/devel/actions). - - juju action do kubernetes-master/0 - -> Note: this example includes curl statements to exercise the app, which -> automatically generates "petstore" transactions written to redis, and allows -> you to visualize the throughput in your browser. - -## Tear down cluster - - ./kube-down.sh - -or destroy your current Juju environment (using the `juju env` command): - - juju destroy-environment --force `juju env` - - -## More Info - -The Kubernetes charms and bundles can be found in the `kubernetes` project on -github.com: - - - [Bundle Repository](http://releases.k8s.io/release-1.1/cluster/juju/bundles) - * [Kubernetes master charm](https://releases.k8s.io/release-1.1/cluster/juju/charms/trusty/kubernetes-master) - * [Kubernetes node charm](https://releases.k8s.io/release-1.1/cluster/juju/charms/trusty/kubernetes) - - [More about Juju](https://jujucharms.com) - - -### Cloud compatibility - -Juju runs natively against a variety of public cloud providers. Juju currently -works with [Amazon Web Service](https://jujucharms.com/docs/stable/config-aws), -[Windows Azure](https://jujucharms.com/docs/stable/config-azure), -[DigitalOcean](https://jujucharms.com/docs/stable/config-digitalocean), -[Google Compute Engine](https://jujucharms.com/docs/stable/config-gce), -[HP Public Cloud](https://jujucharms.com/docs/stable/config-hpcloud), -[Joyent](https://jujucharms.com/docs/stable/config-joyent), -[LXC](https://jujucharms.com/docs/stable/config-LXC), any -[OpenStack](https://jujucharms.com/docs/stable/config-openstack) deployment, -[Vagrant](https://jujucharms.com/docs/stable/config-vagrant), and -[Vmware vSphere](https://jujucharms.com/docs/stable/config-vmware). - -If you do not see your favorite cloud provider listed many clouds can be -configured for [manual provisioning](https://jujucharms.com/docs/stable/config-manual). - -The Kubernetes bundle has been tested on GCE and AWS and found to work with +--- +title: "Getting started with Juju" +--- +[Juju](https://jujucharms.com/docs/stable/about-juju) makes it easy to deploy +Kubernetes by provisioning, installing and configuring all the systems in +the cluster. Once deployed the cluster can easily scale up with one command +to increase the cluster size. + + + +{% include pagetoc.html %} + +## Prerequisites + +> Note: If you're running kube-up, on Ubuntu - all of the dependencies +> will be handled for you. You may safely skip to the section: +> [Launch Kubernetes Cluster](#launch-kubernetes-cluster) + +### On Ubuntu + +[Install the Juju client](https://jujucharms.com/get-started) on your +local Ubuntu system: + + sudo add-apt-repository ppa:juju/stable + sudo apt-get update + sudo apt-get install juju-core juju-quickstart + + +### With Docker + +If you are not using Ubuntu or prefer the isolation of Docker, you may +run the following: + + mkdir ~/.juju + sudo docker run -v ~/.juju:/home/ubuntu/.juju -ti jujusolutions/jujubox:latest + +At this point from either path you will have access to the `juju +quickstart` command. + +To set up the credentials for your chosen cloud run: + + juju quickstart --constraints="mem=3.75G" -i + +> The `constraints` flag is optional, it changes the size of virtual machines +> that Juju will generate when it requests a new machine. Larger machines +> will run faster but cost more money than smaller machines. + +Follow the dialogue and choose `save` and `use`. Quickstart will now +bootstrap the juju root node and setup the juju web based user +interface. + + +## Launch Kubernetes cluster + +You will need to export the `KUBERNETES_PROVIDER` environment variable before +bringing up the cluster. + + export KUBERNETES_PROVIDER=juju + cluster/kube-up.sh + +If this is your first time running the `kube-up.sh` script, it will install +the required dependencies to get started with Juju, additionally it will +launch a curses based configuration utility allowing you to select your cloud +provider and enter the proper access credentials. + +Next it will deploy the kubernetes master, etcd, 2 nodes with flannel based +Software Defined Networking (SDN) so containers on different hosts can +communicate with each other. + + +## Exploring the cluster + +The `juju status` command provides information about each unit in the cluster: + + $ juju status --format=oneline + - docker/0: 52.4.92.78 (started) + - flannel-docker/0: 52.4.92.78 (started) + - kubernetes/0: 52.4.92.78 (started) + - docker/1: 52.6.104.142 (started) + - flannel-docker/1: 52.6.104.142 (started) + - kubernetes/1: 52.6.104.142 (started) + - etcd/0: 52.5.216.210 (started) 4001/tcp + - juju-gui/0: 52.5.205.174 (started) 80/tcp, 443/tcp + - kubernetes-master/0: 52.6.19.238 (started) 8080/tcp + +You can use `juju ssh` to access any of the units: + + juju ssh kubernetes-master/0 + + +## Run some containers! + +`kubectl` is available on the Kubernetes master node. We'll ssh in to +launch some containers, but one could use `kubectl` locally by setting +`KUBERNETES_MASTER` to point at the ip address of "kubernetes-master/0". + +No pods will be available before starting a container: + + kubectl get pods + NAME READY STATUS RESTARTS AGE + + kubectl get replicationcontrollers + CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS + +We'll follow the aws-coreos example. Create a pod manifest: `pod.json` + +{% highlight json %} +{ + "apiVersion": "v1", + "kind": "Pod", + "metadata": { + "name": "hello", + "labels": { + "name": "hello", + "environment": "testing" + } + }, + "spec": { + "containers": [{ + "name": "hello", + "image": "quay.io/kelseyhightower/hello", + "ports": [{ + "containerPort": 80, + "hostPort": 80 + }] + }] + } +} +{% endhighlight %} + +Create the pod with kubectl: + + kubectl create -f pod.json + + +Get info on the pod: + + kubectl get pods + + +To test the hello app, we need to locate which node is hosting +the container. Better tooling for using Juju to introspect container +is in the works but we can use `juju run` and `juju status` to find +our hello app. + +Exit out of our ssh session and run: + + juju run --unit kubernetes/0 "docker ps -n=1" + ... + juju run --unit kubernetes/1 "docker ps -n=1" + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + 02beb61339d8 quay.io/kelseyhightower/hello:latest /hello About an hour ago Up About an hour k8s_hello.... + + +We see "kubernetes/1" has our container, we can open port 80: + + juju run --unit kubernetes/1 "open-port 80" + juju expose kubernetes + sudo apt-get install curl + curl $(juju status --format=oneline kubernetes/1 | cut -d' ' -f3) + +Finally delete the pod: + + juju ssh kubernetes-master/0 + kubectl delete pods hello + + +## Scale out cluster + +We can add node units like so: + + juju add-unit docker # creates unit docker/2, kubernetes/2, docker-flannel/2 + +## Launch the "k8petstore" example app + +The [k8petstore example](../../examples/k8petstore/) is available as a +[juju action](https://jujucharms.com/docs/devel/actions). + + juju action do kubernetes-master/0 + +> Note: this example includes curl statements to exercise the app, which +> automatically generates "petstore" transactions written to redis, and allows +> you to visualize the throughput in your browser. + +## Tear down cluster + + ./kube-down.sh + +or destroy your current Juju environment (using the `juju env` command): + + juju destroy-environment --force `juju env` + + +## More Info + +The Kubernetes charms and bundles can be found in the `kubernetes` project on +github.com: + + - [Bundle Repository](http://releases.k8s.io/release-1.1/cluster/juju/bundles) + * [Kubernetes master charm](https://releases.k8s.io/release-1.1/cluster/juju/charms/trusty/kubernetes-master) + * [Kubernetes node charm](https://releases.k8s.io/release-1.1/cluster/juju/charms/trusty/kubernetes) + - [More about Juju](https://jujucharms.com) + + +### Cloud compatibility + +Juju runs natively against a variety of public cloud providers. Juju currently +works with [Amazon Web Service](https://jujucharms.com/docs/stable/config-aws), +[Windows Azure](https://jujucharms.com/docs/stable/config-azure), +[DigitalOcean](https://jujucharms.com/docs/stable/config-digitalocean), +[Google Compute Engine](https://jujucharms.com/docs/stable/config-gce), +[HP Public Cloud](https://jujucharms.com/docs/stable/config-hpcloud), +[Joyent](https://jujucharms.com/docs/stable/config-joyent), +[LXC](https://jujucharms.com/docs/stable/config-LXC), any +[OpenStack](https://jujucharms.com/docs/stable/config-openstack) deployment, +[Vagrant](https://jujucharms.com/docs/stable/config-vagrant), and +[Vmware vSphere](https://jujucharms.com/docs/stable/config-vmware). + +If you do not see your favorite cloud provider listed many clouds can be +configured for [manual provisioning](https://jujucharms.com/docs/stable/config-manual). + +The Kubernetes bundle has been tested on GCE and AWS and found to work with version 1.0.0. \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/libvirt-coreos.md b/v1.1/docs/getting-started-guides/libvirt-coreos.md index 3a100c1102..25c21ee0f9 100644 --- a/v1.1/docs/getting-started-guides/libvirt-coreos.md +++ b/v1.1/docs/getting-started-guides/libvirt-coreos.md @@ -1,336 +1,287 @@ ---- -title: "Getting started with libvirt CoreOS" -section: guides ---- -## Table of Contents - -{% include pagetoc.html %} - -### Highlights - -* Super-fast cluster boot-up (few seconds instead of several minutes for vagrant) -* Reduced disk usage thanks to [COW](https://en.wikibooks.org/wiki/QEMU/Images#Copy_on_write) -* Reduced memory footprint thanks to [KSM](https://www.kernel.org/doc/Documentation/vm/ksm.txt) - -### Warnings about `libvirt-coreos` use case - -The primary goal of the `libvirt-coreos` cluster provider is to deploy a multi-node Kubernetes cluster on local VMs as fast as possible and to be as light as possible in term of resources used. - -In order to achieve that goal, its deployment is very different from the “standard production deployment†method used on other providers. This was done on purpose in order to implement some optimizations made possible by the fact that we know that all VMs will be running on the same physical machine. - -The `libvirt-coreos` cluster provider doesn’t aim at being production look-alike. - -Another difference is that no security is enforced on `libvirt-coreos` at all. For example, - -* Kube API server is reachable via a clear-text connection (no SSL); -* Kube API server requires no credentials; -* etcd access is not protected; -* Kubernetes secrets are not protected as securely as they are on production environments; -* etc. - -So, an k8s application developer should not validate its interaction with Kubernetes on `libvirt-coreos` because he might technically succeed in doing things that are prohibited on a production environment like: - -* un-authenticated access to Kube API server; -* Access to Kubernetes private data structures inside etcd; -* etc. - -On the other hand, `libvirt-coreos` might be useful for people investigating low level implementation of Kubernetes because debugging techniques like sniffing the network traffic or introspecting the etcd content are easier on `libvirt-coreos` than on a production deployment. - -### Prerequisites - -1. Install [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html) -2. Install [ebtables](http://ebtables.netfilter.org/) -3. Install [qemu](http://wiki.qemu.org/Main_Page) -4. Install [libvirt](http://libvirt.org/) -5. Enable and start the libvirt daemon, e.g: - * ``systemctl enable libvirtd`` - * ``systemctl start libvirtd`` -6. [Grant libvirt access to your user¹](https://libvirt.org/aclpolkit.html) -7. Check that your $HOME is accessible to the qemu user² - -#### ¹ Depending on your distribution, libvirt access may be denied by default or may require a password at each access. - -You can test it with the following command: - -{% highlight sh %} -{% raw %} -virsh -c qemu:///system pool-list -{% endraw %} -{% endhighlight %} - -If you have access error messages, please read https://libvirt.org/acl.html and https://libvirt.org/aclpolkit.html . - -In short, if your libvirt has been compiled with Polkit support (ex: Arch, Fedora 21), you can create `/etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules` as follows to grant full access to libvirt to `$USER` - -{% highlight sh %} -{% raw %} -sudo /bin/sh -c "cat - > /etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules" << EOF -polkit.addRule(function(action, subject) { - if (action.id == "org.libvirt.unix.manage" && - subject.user == "$USER") { - return polkit.Result.YES; - polkit.log("action=" + action); - polkit.log("subject=" + subject); - } -}); -EOF -{% endraw %} -{% endhighlight %} - -If your libvirt has not been compiled with Polkit (ex: Ubuntu 14.04.1 LTS), check the permissions on the libvirt unix socket: - -{% highlight console %} -{% raw %} -$ ls -l /var/run/libvirt/libvirt-sock -srwxrwx--- 1 root libvirtd 0 févr. 12 16:03 /var/run/libvirt/libvirt-sock - -$ usermod -a -G libvirtd $USER -# $USER needs to logout/login to have the new group be taken into account -{% endraw %} -{% endhighlight %} - -(Replace `$USER` with your login name) - -#### ² Qemu will run with a specific user. It must have access to the VMs drives - -All the disk drive resources needed by the VM (CoreOS disk image, Kubernetes binaries, cloud-init files, etc.) are put inside `./cluster/libvirt-coreos/libvirt_storage_pool`. - -As we’re using the `qemu:///system` instance of libvirt, qemu will run with a specific `user:group` distinct from your user. It is configured in `/etc/libvirt/qemu.conf`. That qemu user must have access to that libvirt storage pool. - -If your `$HOME` is world readable, everything is fine. If your $HOME is private, `cluster/kube-up.sh` will fail with an error message like: - -{% highlight console %} -{% raw %} -error: Cannot access storage file '$HOME/.../kubernetes/cluster/libvirt-coreos/libvirt_storage_pool/kubernetes_master.img' (as uid:99, gid:78): Permission denied -{% endraw %} -{% endhighlight %} - -In order to fix that issue, you have several possibilities: -* set `POOL_PATH` inside `cluster/libvirt-coreos/config-default.sh` to a directory: - * backed by a filesystem with a lot of free disk space - * writable by your user; - * accessible by the qemu user. -* Grant the qemu user access to the storage pool. - -On Arch: - -{% highlight sh %} -{% raw %} -setfacl -m g:kvm:--x ~ -{% endraw %} -{% endhighlight %} - -### Setup - -By default, the libvirt-coreos setup will create a single Kubernetes master and 3 Kubernetes nodes. Because the VM drives use Copy-on-Write and because of memory ballooning and KSM, there is a lot of resource over-allocation. - -To start your local cluster, open a shell and run: - -{% highlight sh %} -{% raw %} -cd kubernetes - -export KUBERNETES_PROVIDER=libvirt-coreos -cluster/kube-up.sh -{% endraw %} -{% endhighlight %} - -The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster management scripts which variant to use. If you forget to set this, the assumption is you are running on Google Compute Engine. - -The `NUM_MINIONS` environment variable may be set to specify the number of nodes to start. If it is not set, the number of nodes defaults to 3. - -The `KUBE_PUSH` environment variable may be set to specify which Kubernetes binaries must be deployed on the cluster. Its possible values are: - -* `release` (default if `KUBE_PUSH` is not set) will deploy the binaries of `_output/release-tars/kubernetes-server-….tar.gz`. This is built with `make release` or `make release-skip-tests`. -* `local` will deploy the binaries of `_output/local/go/bin`. These are built with `make`. - -You can check that your machines are there and running with: - -{% highlight console %} -{% raw %} -$ virsh -c qemu:///system list - Id Name State ----------------------------------------------------- - 15 kubernetes_master running - 16 kubernetes_minion-01 running - 17 kubernetes_minion-02 running - 18 kubernetes_minion-03 running -{% endraw %} -{% endhighlight %} - -You can check that the Kubernetes cluster is working with: - -{% highlight console %} -{% raw %} -$ kubectl get nodes -NAME LABELS STATUS -192.168.10.2 Ready -192.168.10.3 Ready -192.168.10.4 Ready -{% endraw %} -{% endhighlight %} - -The VMs are running [CoreOS](https://coreos.com/). -Your ssh keys have already been pushed to the VM. (It looks for ~/.ssh/id_*.pub) -The user to use to connect to the VM is `core`. -The IP to connect to the master is 192.168.10.1. -The IPs to connect to the nodes are 192.168.10.2 and onwards. - -Connect to `kubernetes_master`: - -{% highlight sh %} -{% raw %} -ssh core@192.168.10.1 -{% endraw %} -{% endhighlight %} - -Connect to `kubernetes_minion-01`: - -{% highlight sh %} -{% raw %} -ssh core@192.168.10.2 -{% endraw %} -{% endhighlight %} - -### Interacting with your Kubernetes cluster with the `kube-*` scripts. - -All of the following commands assume you have set `KUBERNETES_PROVIDER` appropriately: - -{% highlight sh %} -{% raw %} -export KUBERNETES_PROVIDER=libvirt-coreos -{% endraw %} -{% endhighlight %} - -Bring up a libvirt-CoreOS cluster of 5 nodes - -{% highlight sh %} -{% raw %} -NUM_MINIONS=5 cluster/kube-up.sh -{% endraw %} -{% endhighlight %} - -Destroy the libvirt-CoreOS cluster - -{% highlight sh %} -{% raw %} -cluster/kube-down.sh -{% endraw %} -{% endhighlight %} - -Update the libvirt-CoreOS cluster with a new Kubernetes release produced by `make release` or `make release-skip-tests`: - -{% highlight sh %} -{% raw %} -cluster/kube-push.sh -{% endraw %} -{% endhighlight %} - -Update the libvirt-CoreOS cluster with the locally built Kubernetes binaries produced by `make`: - -{% highlight sh %} -{% raw %} -KUBE_PUSH=local cluster/kube-push.sh -{% endraw %} -{% endhighlight %} - -Interact with the cluster - -{% highlight sh %} -{% raw %} -kubectl ... -{% endraw %} -{% endhighlight %} - -### Troubleshooting - -#### !!! Cannot find kubernetes-server-linux-amd64.tar.gz - -Build the release tarballs: - -{% highlight sh %} -{% raw %} -make release -{% endraw %} -{% endhighlight %} - -#### Can't find virsh in PATH, please fix and retry. - -Install libvirt - -On Arch: - -{% highlight sh %} -{% raw %} -pacman -S qemu libvirt -{% endraw %} -{% endhighlight %} - -On Ubuntu 14.04.1: - -{% highlight sh %} -{% raw %} -aptitude install qemu-system-x86 libvirt-bin -{% endraw %} -{% endhighlight %} - -On Fedora 21: - -{% highlight sh %} -{% raw %} -yum install qemu libvirt -{% endraw %} -{% endhighlight %} - -#### error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory - -Start the libvirt daemon - -On Arch: - -{% highlight sh %} -{% raw %} -systemctl start libvirtd -{% endraw %} -{% endhighlight %} - -On Ubuntu 14.04.1: - -{% highlight sh %} -{% raw %} -service libvirt-bin start -{% endraw %} -{% endhighlight %} - -#### error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied - -Fix libvirt access permission (Remember to adapt `$USER`) - -On Arch and Fedora 21: - -{% highlight sh %} -{% raw %} -cat > /etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules < /etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules" << EOF +polkit.addRule(function(action, subject) { + if (action.id == "org.libvirt.unix.manage" && + subject.user == "$USER") { + return polkit.Result.YES; + polkit.log("action=" + action); + polkit.log("subject=" + subject); + } +}); +EOF +{% endhighlight %} + +If your libvirt has not been compiled with Polkit (ex: Ubuntu 14.04.1 LTS), check the permissions on the libvirt unix socket: + +{% raw %} +{% highlight console %} +$ ls -l /var/run/libvirt/libvirt-sock +srwxrwx--- 1 root libvirtd 0 févr. 12 16:03 /var/run/libvirt/libvirt-sock + +$ usermod -a -G libvirtd $USER +# $USER needs to logout/login to have the new group be taken into account +{% endhighlight %} +{% endraw %} + +(Replace `$USER` with your login name) + +#### ² Qemu will run with a specific user. It must have access to the VMs drives + +All the disk drive resources needed by the VM (CoreOS disk image, Kubernetes binaries, cloud-init files, etc.) are put inside `./cluster/libvirt-coreos/libvirt_storage_pool`. + +As we're using the `qemu:///system` instance of libvirt, qemu will run with a specific `user:group` distinct from your user. It is configured in `/etc/libvirt/qemu.conf`. That qemu user must have access to that libvirt storage pool. + +If your `$HOME` is world readable, everything is fine. If your $HOME is private, `cluster/kube-up.sh` will fail with an error message like: + +{% highlight console %} +error: Cannot access storage file '$HOME/.../kubernetes/cluster/libvirt-coreos/libvirt_storage_pool/kubernetes_master.img' (as uid:99, gid:78): Permission denied +{% endhighlight %} + +In order to fix that issue, you have several possibilities: +* set `POOL_PATH` inside `cluster/libvirt-coreos/config-default.sh` to a directory: + * backed by a filesystem with a lot of free disk space + * writable by your user; + * accessible by the qemu user. +* Grant the qemu user access to the storage pool. + +On Arch: + +{% highlight sh %} +setfacl -m g:kvm:--x ~ +{% endhighlight %} + +### Setup + +By default, the libvirt-coreos setup will create a single Kubernetes master and 3 Kubernetes nodes. Because the VM drives use Copy-on-Write and because of memory ballooning and KSM, there is a lot of resource over-allocation. + +To start your local cluster, open a shell and run: + +{% highlight sh %} +cd kubernetes + +export KUBERNETES_PROVIDER=libvirt-coreos +cluster/kube-up.sh +{% endhighlight %} + +The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster management scripts which variant to use. If you forget to set this, the assumption is you are running on Google Compute Engine. + +The `NUM_MINIONS` environment variable may be set to specify the number of nodes to start. If it is not set, the number of nodes defaults to 3. + +The `KUBE_PUSH` environment variable may be set to specify which Kubernetes binaries must be deployed on the cluster. Its possible values are: + +* `release` (default if `KUBE_PUSH` is not set) will deploy the binaries of `_output/release-tars/kubernetes-server-'|.tar.gz`. This is built with `make release` or `make release-skip-tests`. +* `local` will deploy the binaries of `_output/local/go/bin`. These are built with `make`. + +You can check that your machines are there and running with: + +{% highlight console %} +$ virsh -c qemu:///system list + Id Name State +---------------------------------------------------- + 15 kubernetes_master running + 16 kubernetes_minion-01 running + 17 kubernetes_minion-02 running + 18 kubernetes_minion-03 running +{% endhighlight %} + +You can check that the Kubernetes cluster is working with: + +{% highlight console %} +$ kubectl get nodes +NAME LABELS STATUS +192.168.10.2 Ready +192.168.10.3 Ready +192.168.10.4 Ready +{% endhighlight %} + +The VMs are running [CoreOS](https://coreos.com/). +Your ssh keys have already been pushed to the VM. (It looks for ~/.ssh/id_*.pub) +The user to use to connect to the VM is `core`. +The IP to connect to the master is 192.168.10.1. +The IPs to connect to the nodes are 192.168.10.2 and onwards. + +Connect to `kubernetes_master`: + +{% highlight sh %} +ssh core@192.168.10.1 +{% endhighlight %} + +Connect to `kubernetes_minion-01`: + +{% highlight sh %} +ssh core@192.168.10.2 +{% endhighlight %} + +### Interacting with your Kubernetes cluster with the `kube-*` scripts. + +All of the following commands assume you have set `KUBERNETES_PROVIDER` appropriately: + +{% highlight sh %} +export KUBERNETES_PROVIDER=libvirt-coreos +{% endhighlight %} + +Bring up a libvirt-CoreOS cluster of 5 nodes + +{% highlight sh %} +NUM_MINIONS=5 cluster/kube-up.sh +{% endhighlight %} + +Destroy the libvirt-CoreOS cluster + +{% highlight sh %} +cluster/kube-down.sh +{% endhighlight %} + +Update the libvirt-CoreOS cluster with a new Kubernetes release produced by `make release` or `make release-skip-tests`: + +{% highlight sh %} +cluster/kube-push.sh +{% endhighlight %} + +Update the libvirt-CoreOS cluster with the locally built Kubernetes binaries produced by `make`: + +{% highlight sh %} +KUBE_PUSH=local cluster/kube-push.sh +{% endhighlight %} + +Interact with the cluster + +{% highlight sh %} +kubectl ... +{% endhighlight %} + +### Troubleshooting + +#### !!! Cannot find kubernetes-server-linux-amd64.tar.gz + +Build the release tarballs: + +{% highlight sh %} +make release +{% endhighlight %} + +#### Can't find virsh in PATH, please fix and retry. + +Install libvirt + +On Arch: + +{% highlight sh %} +pacman -S qemu libvirt +{% endhighlight %} + +On Ubuntu 14.04.1: + +{% highlight sh %} +aptitude install qemu-system-x86 libvirt-bin +{% endhighlight %} + +On Fedora 21: + +{% highlight sh %} +yum install qemu libvirt +{% endhighlight %} + +#### error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory + +Start the libvirt daemon + +On Arch: + +{% highlight sh %} +systemctl start libvirtd +{% endhighlight %} + +On Ubuntu 14.04.1: + +{% highlight sh %} +service libvirt-bin start +{% endhighlight %} + +#### error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied + +Fix libvirt access permission (Remember to adapt `$USER`) + +On Arch and Fedora 21: + +{% highlight sh %} +cat > /etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules < {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: @@ -46,30 +42,30 @@ spec: image: ubuntu:14.04 args: [bash, -c, 'for ((i = 0; ; i++)); do echo "$i: $(date)"; sleep 1; done'] -{% endraw %} + {% endhighlight %} [Download example](../../examples/blog-logging/counter-pod.yaml) -This pod specification has one container which runs a bash script when the container is born. This script simply writes out the value of a counter and the date once per second and runs indefinitely. Let’s create the pod in the default +This pod specification has one container which runs a bash script when the container is born. This script simply writes out the value of a counter and the date once per second and runs indefinitely. Let's create the pod in the default namespace. {% highlight console %} -{% raw %} + $ kubectl create -f examples/blog-logging/counter-pod.yaml pods/counter -{% endraw %} + {% endhighlight %} We can observe the running pod: {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY STATUS RESTARTS AGE counter 1/1 Running 0 5m -{% endraw %} + {% endhighlight %} This step may take a few minutes to download the ubuntu:14.04 image during which the pod status will be shown as `Pending`. @@ -81,7 +77,7 @@ One of the nodes is now running the counter pod: When the pod status changes to `Running` we can use the kubectl logs command to view the output of this counter pod. {% highlight console %} -{% raw %} + $ kubectl logs counter 0: Tue Jun 2 21:37:31 UTC 2015 1: Tue Jun 2 21:37:32 UTC 2015 @@ -90,13 +86,13 @@ $ kubectl logs counter 4: Tue Jun 2 21:37:35 UTC 2015 5: Tue Jun 2 21:37:36 UTC 2015 ... -{% endraw %} + {% endhighlight %} This command fetches the log text from the Docker log file for the image that is running in this container. We can connect to the running container and observe the running counter bash script. {% highlight console %} -{% raw %} + $ kubectl exec -i counter bash ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND @@ -104,31 +100,31 @@ root 1 0.0 0.0 17976 2888 ? Ss 00:02 0:00 bash -c for ((i root 468 0.0 0.0 17968 2904 ? Ss 00:05 0:00 bash root 479 0.0 0.0 4348 812 ? S 00:05 0:00 sleep 1 root 480 0.0 0.0 15572 2212 ? R 00:05 0:00 ps aux -{% endraw %} + {% endhighlight %} -What happens if for any reason the image in this pod is killed off and then restarted by Kubernetes? Will we still see the log lines from the previous invocation of the container followed by the log lines for the started container? Or will we lose the log lines from the original container’s execution and only see the log lines for the new container? Let’s find out. First let’s stop the currently running counter. +What happens if for any reason the image in this pod is killed off and then restarted by Kubernetes? Will we still see the log lines from the previous invocation of the container followed by the log lines for the started container? Or will we lose the log lines from the original container's execution and only see the log lines for the new container? Let's find out. First let's stop the currently running counter. {% highlight console %} -{% raw %} + $ kubectl stop pod counter pods/counter -{% endraw %} + {% endhighlight %} -Now let’s restart the counter. +Now let's restart the counter. {% highlight console %} -{% raw %} + $ kubectl create -f examples/blog-logging/counter-pod.yaml pods/counter -{% endraw %} + {% endhighlight %} -Let’s wait for the container to restart and get the log lines again. +Let's wait for the container to restart and get the log lines again. {% highlight console %} -{% raw %} + $ kubectl logs counter 0: Tue Jun 2 21:51:40 UTC 2015 1: Tue Jun 2 21:51:41 UTC 2015 @@ -139,10 +135,10 @@ $ kubectl logs counter 6: Tue Jun 2 21:51:46 UTC 2015 7: Tue Jun 2 21:51:47 UTC 2015 8: Tue Jun 2 21:51:48 UTC 2015 -{% endraw %} + {% endhighlight %} -We’ve lost the log lines from the first invocation of the container in this pod! Ideally, we want to preserve all the log lines from each invocation of each container in the pod. Furthermore, even if the pod is restarted we would still like to preserve all the log lines that were ever emitted by the containers in the pod. But don’t fear, this is the functionality provided by cluster level logging in Kubernetes. When a cluster is created, the standard output and standard error output of each container can be ingested using a [Fluentd](http://www.fluentd.org/) agent running on each node into either [Google Cloud Logging](https://cloud.google.com/logging/docs/) or into Elasticsearch and viewed with Kibana. +We've lost the log lines from the first invocation of the container in this pod! Ideally, we want to preserve all the log lines from each invocation of each container in the pod. Furthermore, even if the pod is restarted we would still like to preserve all the log lines that were ever emitted by the containers in the pod. But don't fear, this is the functionality provided by cluster level logging in Kubernetes. When a cluster is created, the standard output and standard error output of each container can be ingested using a [Fluentd](http://www.fluentd.org/) agent running on each node into either [Google Cloud Logging](https://cloud.google.com/logging/docs/) or into Elasticsearch and viewed with Kibana. When a Kubernetes cluster is created with logging to Google Cloud Logging enabled, the system creates a pod called `fluentd-cloud-logging` on each node of the cluster to collect Docker container logs. These pods were shown at the start of this blog article in the response to the first get pods command. @@ -151,7 +147,7 @@ This log collection pod has a specification which looks something like this: {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: @@ -182,7 +178,7 @@ spec: - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers -{% endraw %} + {% endhighlight %} [Download example](https://releases.k8s.io/release-1.1/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml) @@ -206,11 +202,11 @@ Note the first container counted to 108 and then it was terminated. When the nex We could query the ingested logs from BigQuery using the SQL query which reports the counter log lines showing the newest lines first: {% highlight console %} -{% raw %} + SELECT metadata.timestamp, structPayload.log FROM [mylogs.kubernetes_counter_default_count_20150611] ORDER BY metadata.timestamp DESC -{% endraw %} + {% endhighlight %} Here is some sample output: @@ -221,15 +217,15 @@ We could also fetch the logs from Google Cloud Storage buckets to our desktop or {% highlight console %} -{% raw %} + $ gsutil -m cp -r gs://myproject/kubernetes.counter_default_count/2015/06/11 . -{% endraw %} + {% endhighlight %} Now we can run queries over the ingested logs. The example below uses the [jq](http://stedolan.github.io/jq/) program to extract just the log lines. {% highlight console %} -{% raw %} + $ cat 21\:00\:00_21\:59\:59_S0.json | jq '.structPayload.log' "0: Thu Jun 11 21:39:38 UTC 2015\n" "1: Thu Jun 11 21:39:39 UTC 2015\n" @@ -240,12 +236,12 @@ $ cat 21\:00\:00_21\:59\:59_S0.json | jq '.structPayload.log' "6: Thu Jun 11 21:39:44 UTC 2015\n" "7: Thu Jun 11 21:39:45 UTC 2015\n" ... -{% endraw %} + {% endhighlight %} -This page has touched briefly on the underlying mechanisms that support gathering cluster level logs on a Kubernetes deployment. The approach here only works for gathering the standard output and standard error output of the processes running in the pod’s containers. To gather other logs that are stored in files one can use a sidecar container to gather the required files as described at the page [Collecting log files within containers with Fluentd](http://releases.k8s.io/release-1.1/contrib/logging/fluentd-sidecar-gcp/README.md) and sending them to the Google Cloud Logging service. +This page has touched briefly on the underlying mechanisms that support gathering cluster level logs on a Kubernetes deployment. The approach here only works for gathering the standard output and standard error output of the processes running in the pod's containers. To gather other logs that are stored in files one can use a sidecar container to gather the required files as described at the page [Collecting log files within containers with Fluentd](http://releases.k8s.io/release-1.1/contrib/logging/fluentd-sidecar-gcp/README.md) and sending them to the Google Cloud Logging service. -Some of the material in this section also appears in the blog article [Cluster Level Logging with Kubernetes](http://blog.kubernetes.io/2015/06/cluster-level-logging-with-kubernetes.html). +Some of the material in this section also appears in the blog article [Cluster Level Logging with Kubernetes](http://blog.kubernetes.io/2015/06/cluster-level-logging-with-kubernetes). diff --git a/v1.1/docs/getting-started-guides/mesos-docker.md b/v1.1/docs/getting-started-guides/mesos-docker.md index 4c9e2941b8..b0fca86f0a 100644 --- a/v1.1/docs/getting-started-guides/mesos-docker.md +++ b/v1.1/docs/getting-started-guides/mesos-docker.md @@ -1,12 +1,11 @@ --- title: "Getting Started With Kubernetes on Mesos on Docker" -section: samples --- The mesos/docker provider uses docker-compose to launch Kubernetes as a Mesos framework, running in docker with its dependencies (etcd & mesos). -## Table of Contents + {% include pagetoc.html %} @@ -163,7 +162,7 @@ host machine (mac). If the configured resources are not available on the host, you may want to increase the resources available to Docker Engine. You may have to increase you VM disk, memory, or cpu allocation in VirtualBox, [Docker Machine](https://docs.docker.com/machine/#oracle-virtualbox), or - [Boot2Docker](https://ryanfb.github.io/etc/2015/01/28/increasing_boot2docker_allocations_on_os_x.html). + [Boot2Docker](https://ryanfb.github.io/etc/2015/01/28/increasing_boot2docker_allocations_on_os_x). 1. Configure provider diff --git a/v1.1/docs/getting-started-guides/mesos.md b/v1.1/docs/getting-started-guides/mesos.md index 18953c056e..5e2814bbbd 100644 --- a/v1.1/docs/getting-started-guides/mesos.md +++ b/v1.1/docs/getting-started-guides/mesos.md @@ -1,406 +1,344 @@ ---- -title: "Getting started with Kubernetes on Mesos" ---- - - -Getting started with Kubernetes on Mesos ----------------------------------------- - -**Table of Contents** - - - - [About Kubernetes on Mesos](#about-kubernetes-on-mesos) - - [Prerequisites](#prerequisites) - - [Deploy Kubernetes-Mesos](#deploy-kubernetes-mesos) - - [Deploy etcd](#deploy-etcd) - - [Start Kubernetes-Mesos Services](#start-kubernetes-mesos-services) - - [Validate KM Services](#validate-km-services) - - [Spin up a pod](#spin-up-a-pod) - - [Launching kube-dns](#launching-kube-dns) - - [What next?](#what-next) - - - - -## About Kubernetes on Mesos - - - -Mesos allows dynamic sharing of cluster resources between Kubernetes and other first-class Mesos frameworks such as [Hadoop][1], [Spark][2], and [Chronos][3]. -Mesos also ensures applications from different frameworks running on your cluster are isolated and that resources are allocated fairly among them. - -Mesos clusters can be deployed on nearly every IaaS cloud provider infrastructure or in your own physical datacenter. Kubernetes on Mesos runs on-top of that and therefore allows you to easily move Kubernetes workloads from one of these environments to the other. - -This tutorial will walk you through setting up Kubernetes on a Mesos cluster. -It provides a step by step walk through of adding Kubernetes to a Mesos cluster and starting your first pod with an nginx webserver. - -**NOTE:** There are [known issues with the current implementation][7] and support for centralized logging and monitoring is not yet available. -Please [file an issue against the kubernetes-mesos project][8] if you have problems completing the steps below. - -Further information is available in the Kubernetes on Mesos [contrib directory][13]. - -### Prerequisites - -- Understanding of [Apache Mesos][6] -- A running [Mesos cluster on Google Compute Engine][5] -- A [VPN connection][10] to the cluster -- A machine in the cluster which should become the Kubernetes *master node* with: - - GoLang > 1.2 - - make (i.e. build-essential) - - Docker - -**Note**: You *can*, but you *don't have to* deploy Kubernetes-Mesos on the same machine the Mesos master is running on. - -### Deploy Kubernetes-Mesos - -Log into the future Kubernetes *master node* over SSH, replacing the placeholder below with the correct IP address. - -{% highlight bash %} -{% raw %} -ssh jclouds@${ip_address_of_master_node} -{% endraw %} -{% endhighlight %} - -Build Kubernetes-Mesos. - -{% highlight bash %} -{% raw %} -git clone https://github.com/kubernetes/kubernetes -cd kubernetes -export KUBERNETES_CONTRIB=mesos -make -{% endraw %} -{% endhighlight %} - -Set some environment variables. -The internal IP address of the master may be obtained via `hostname -i`. - -{% highlight bash %} -{% raw %} -export KUBERNETES_MASTER_IP=$(hostname -i) -export KUBERNETES_MASTER=http://${KUBERNETES_MASTER_IP}:8888 -{% endraw %} -{% endhighlight %} - -Note that KUBERNETES_MASTER is used as the api endpoint. If you have existing `~/.kube/config` and point to another endpoint, you need to add option `--server=${KUBERNETES_MASTER}` to kubectl in later steps. - -### Deploy etcd - -Start etcd and verify that it is running: - -{% highlight bash %} -{% raw %} -sudo docker run -d --hostname $(uname -n) --name etcd \ - -p 4001:4001 -p 7001:7001 quay.io/coreos/etcd:v2.0.12 \ - --listen-client-urls http://0.0.0.0:4001 \ - --advertise-client-urls http://${KUBERNETES_MASTER_IP}:4001 -{% endraw %} -{% endhighlight %} - -{% highlight console %} -{% raw %} -$ sudo docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -fd7bac9e2301 quay.io/coreos/etcd:v2.0.12 "/etcd" 5s ago Up 3s 2379/tcp, 2380/... etcd -{% endraw %} -{% endhighlight %} - -It's also a good idea to ensure your etcd instance is reachable by testing it - -{% highlight bash %} -{% raw %} -curl -L http://${KUBERNETES_MASTER_IP}:4001/v2/keys/ -{% endraw %} -{% endhighlight %} - -If connectivity is OK, you will see an output of the available keys in etcd (if any). - -### Start Kubernetes-Mesos Services - -Update your PATH to more easily run the Kubernetes-Mesos binaries: - -{% highlight bash %} -{% raw %} -export PATH="$(pwd)/_output/local/go/bin:$PATH" -{% endraw %} -{% endhighlight %} - -Identify your Mesos master: depending on your Mesos installation this is either a `host:port` like `mesos-master:5050` or a ZooKeeper URL like `zk://zookeeper:2181/mesos`. -In order to let Kubernetes survive Mesos master changes, the ZooKeeper URL is recommended for production environments. - -{% highlight bash %} -{% raw %} -export MESOS_MASTER= -{% endraw %} -{% endhighlight %} - -Create a cloud config file `mesos-cloud.conf` in the current directory with the following contents: - -{% highlight console %} -{% raw %} -$ cat <mesos-cloud.conf -[mesos-cloud] - mesos-master = ${MESOS_MASTER} -EOF -{% endraw %} -{% endhighlight %} - -Now start the kubernetes-mesos API server, controller manager, and scheduler on the master node: - -{% highlight console %} -{% raw %} -$ km apiserver \ - --address=${KUBERNETES_MASTER_IP} \ - --etcd-servers=http://${KUBERNETES_MASTER_IP}:4001 \ - --service-cluster-ip-range=10.10.10.0/24 \ - --port=8888 \ - --cloud-provider=mesos \ - --cloud-config=mesos-cloud.conf \ - --secure-port=0 \ - --v=1 >apiserver.log 2>&1 & - -$ km controller-manager \ - --master=${KUBERNETES_MASTER_IP}:8888 \ - --cloud-provider=mesos \ - --cloud-config=./mesos-cloud.conf \ - --v=1 >controller.log 2>&1 & - -$ km scheduler \ - --address=${KUBERNETES_MASTER_IP} \ - --mesos-master=${MESOS_MASTER} \ - --etcd-servers=http://${KUBERNETES_MASTER_IP}:4001 \ - --mesos-user=root \ - --api-servers=${KUBERNETES_MASTER_IP}:8888 \ - --cluster-dns=10.10.10.10 \ - --cluster-domain=cluster.local \ - --v=2 >scheduler.log 2>&1 & -{% endraw %} -{% endhighlight %} - -Disown your background jobs so that they'll stay running if you log out. - -{% highlight bash %} -{% raw %} -disown -a -{% endraw %} -{% endhighlight %} - -#### Validate KM Services - -Add the appropriate binary folder to your `PATH` to access kubectl: - -{% highlight bash %} -{% raw %} -export PATH=/platforms/linux/amd64:$PATH -{% endraw %} -{% endhighlight %} - -Interact with the kubernetes-mesos framework via `kubectl`: - -{% highlight console %} -{% raw %} -$ kubectl get pods -NAME READY STATUS RESTARTS AGE -{% endraw %} -{% endhighlight %} - -{% highlight console %} -{% raw %} -# NOTE: your service IPs will likely differ -$ kubectl get services -NAME LABELS SELECTOR IP(S) PORT(S) -k8sm-scheduler component=scheduler,provider=k8sm 10.10.10.113 10251/TCP -kubernetes component=apiserver,provider=kubernetes 10.10.10.1 443/TCP -{% endraw %} -{% endhighlight %} - -Lastly, look for Kubernetes in the Mesos web GUI by pointing your browser to -`http://`. Make sure you have an active VPN connection. -Go to the Frameworks tab, and look for an active framework named "Kubernetes". - -## Spin up a pod - -Write a JSON pod description to a local file: - -{% highlight bash %} -{% raw %} -$ cat <nginx.yaml -{% endraw %} -{% endhighlight %} - -{% highlight yaml %} -{% raw %} -apiVersion: v1 -kind: Pod -metadata: - name: nginx -spec: - containers: - - name: nginx - image: nginx - ports: - - containerPort: 80 -EOPOD -{% endraw %} -{% endhighlight %} - -Send the pod description to Kubernetes using the `kubectl` CLI: - -{% highlight console %} -{% raw %} -$ kubectl create -f ./nginx.yaml -pods/nginx -{% endraw %} -{% endhighlight %} - -Wait a minute or two while `dockerd` downloads the image layers from the internet. -We can use the `kubectl` interface to monitor the status of our pod: - -{% highlight console %} -{% raw %} -$ kubectl get pods -NAME READY STATUS RESTARTS AGE -nginx 1/1 Running 0 14s -{% endraw %} -{% endhighlight %} - -Verify that the pod task is running in the Mesos web GUI. Click on the -Kubernetes framework. The next screen should show the running Mesos task that -started the Kubernetes pod. - -## Launching kube-dns - -Kube-dns is an addon for Kubernetes which adds DNS-based service discovery to the cluster. For a detailed explanation see [DNS in Kubernetes][4]. - -The kube-dns addon runs as a pod inside the cluster. The pod consists of three co-located containers: - -- a local etcd instance -- the [skydns][11] DNS server -- the kube2sky process to glue skydns to the state of the Kubernetes cluster. - -The skydns container offers DNS service via port 53 to the cluster. The etcd communication works via local 127.0.0.1 communication - -We assume that kube-dns will use - -- the service IP `10.10.10.10` -- and the `cluster.local` domain. - -Note that we have passed these two values already as parameter to the apiserver above. - -A template for an replication controller spinning up the pod with the 3 containers can be found at [cluster/addons/dns/skydns-rc.yaml.in][11] in the repository. The following steps are necessary in order to get a valid replication controller yaml file: - -- replace `{{ pillar['dns_replicas'] }}` with `1` -- replace `{{ pillar['dns_domain'] }}` with `cluster.local.` -- add `--kube_master_url=${KUBERNETES_MASTER}` parameter to the kube2sky container command. - -In addition the service template at [cluster/addons/dns/skydns-svc.yaml.in][12] needs the following replacement: - -- `{{ pillar['dns_server'] }}` with `10.10.10.10`. - -To do this automatically: - -{% highlight bash %} -{% raw %} -sed -e "s/{{ pillar\['dns_replicas'\] }}/1/g;"\ -"s,\(command = \"/kube2sky\"\),\\1\\"$'\n'" - --kube_master_url=${KUBERNETES_MASTER},;"\ -"s/{{ pillar\['dns_domain'\] }}/cluster.local/g" \ - cluster/addons/dns/skydns-rc.yaml.in > skydns-rc.yaml -sed -e "s/{{ pillar\['dns_server'\] }}/10.10.10.10/g" \ - cluster/addons/dns/skydns-svc.yaml.in > skydns-svc.yaml -{% endraw %} -{% endhighlight %} - -Now the kube-dns pod and service are ready to be launched: - -{% highlight bash %} -{% raw %} -kubectl create -f ./skydns-rc.yaml -kubectl create -f ./skydns-svc.yaml -{% endraw %} -{% endhighlight %} - -Check with `kubectl get pods --namespace=kube-system` that 3/3 containers of the pods are eventually up and running. Note that the kube-dns pods run in the `kube-system` namespace, not in `default`. - -To check that the new DNS service in the cluster works, we start a busybox pod and use that to do a DNS lookup. First create the `busybox.yaml` pod spec: - -{% highlight bash %} -{% raw %} -cat <busybox.yaml -{% endraw %} -{% endhighlight %} - -{% highlight yaml %} -{% raw %} -apiVersion: v1 -kind: Pod -metadata: - name: busybox - namespace: default -spec: - containers: - - image: busybox - command: - - sleep - - "3600" - imagePullPolicy: IfNotPresent - name: busybox - restartPolicy: Always -EOF -{% endraw %} -{% endhighlight %} - -Then start the pod: - -{% highlight bash %} -{% raw %} -kubectl create -f ./busybox.yaml -{% endraw %} -{% endhighlight %} - -When the pod is up and running, start a lookup for the Kubernetes master service, made available on 10.10.10.1 by default: - -{% highlight bash %} -{% raw %} -kubectl exec busybox -- nslookup kubernetes -{% endraw %} -{% endhighlight %} - -If everything works fine, you will get this output: - -{% highlight console %} -{% raw %} -Server: 10.10.10.10 -Address 1: 10.10.10.10 - -Name: kubernetes -Address 1: 10.10.10.1 -{% endraw %} -{% endhighlight %} - -## What next? - -Try out some of the standard [Kubernetes examples][9]. - -Read about Kubernetes on Mesos' architecture in the [contrib directory][13]. - -**NOTE:** Some examples require Kubernetes DNS to be installed on the cluster. -Future work will add instructions to this guide to enable support for Kubernetes DNS. - -**NOTE:** Please be aware that there are [known issues with the current Kubernetes-Mesos implementation][7]. - -[1]: http://mesosphere.com/docs/tutorials/run-hadoop-on-mesos-using-installer -[2]: http://mesosphere.com/docs/tutorials/run-spark-on-mesos -[3]: http://mesosphere.com/docs/tutorials/run-chronos-on-mesos -[4]: https://releases.k8s.io/release-1.1/cluster/addons/dns/README.md -[5]: http://open.mesosphere.com/getting-started/cloud/google/mesosphere/ -[6]: http://mesos.apache.org/ -[7]: https://releases.k8s.io/release-1.1/contrib/mesos/docs/issues.md -[8]: https://github.com/mesosphere/kubernetes-mesos/issues -[9]: ../../examples/ -[10]: http://open.mesosphere.com/getting-started/cloud/google/mesosphere/#vpn-setup -[11]: https://releases.k8s.io/release-1.1/cluster/addons/dns/skydns-rc.yaml.in -[12]: https://releases.k8s.io/release-1.1/cluster/addons/dns/skydns-svc.yaml.in -[13]: https://releases.k8s.io/release-1.1/contrib/mesos/README.md - - - +--- +title: "Getting started with Kubernetes on Mesos" +--- + + +Getting started with Kubernetes on Mesos +---------------------------------------- + +{% include pagetoc.html %} + + + +## About Kubernetes on Mesos + + + +Mesos allows dynamic sharing of cluster resources between Kubernetes and other first-class Mesos frameworks such as [Hadoop][1], [Spark][2], and [Chronos][3]. +Mesos also ensures applications from different frameworks running on your cluster are isolated and that resources are allocated fairly among them. + +Mesos clusters can be deployed on nearly every IaaS cloud provider infrastructure or in your own physical datacenter. Kubernetes on Mesos runs on-top of that and therefore allows you to easily move Kubernetes workloads from one of these environments to the other. + +This tutorial will walk you through setting up Kubernetes on a Mesos cluster. +It provides a step by step walk through of adding Kubernetes to a Mesos cluster and starting your first pod with an nginx webserver. + +**NOTE:** There are [known issues with the current implementation][7] and support for centralized logging and monitoring is not yet available. +Please [file an issue against the kubernetes-mesos project][8] if you have problems completing the steps below. + +Further information is available in the Kubernetes on Mesos [contrib directory][13]. + +### Prerequisites + +- Understanding of [Apache Mesos][6] +- A running [Mesos cluster on Google Compute Engine][5] +- A [VPN connection][10] to the cluster +- A machine in the cluster which should become the Kubernetes *master node* with: + - GoLang > 1.2 + - make (i.e. build-essential) + - Docker + +**Note**: You *can*, but you *don't have to* deploy Kubernetes-Mesos on the same machine the Mesos master is running on. + +### Deploy Kubernetes-Mesos + +Log into the future Kubernetes *master node* over SSH, replacing the placeholder below with the correct IP address. + +{% highlight bash %} +ssh jclouds@${ip_address_of_master_node} +{% endhighlight %} + +Build Kubernetes-Mesos. + +{% highlight bash %} +git clone https://github.com/kubernetes/kubernetes +cd kubernetes +export KUBERNETES_CONTRIB=mesos +make +{% endhighlight %} + +Set some environment variables. +The internal IP address of the master may be obtained via `hostname -i`. + +{% highlight bash %} +export KUBERNETES_MASTER_IP=$(hostname -i) +export KUBERNETES_MASTER=http://${KUBERNETES_MASTER_IP}:8888 +{% endhighlight %} + +Note that KUBERNETES_MASTER is used as the api endpoint. If you have existing `~/.kube/config` and point to another endpoint, you need to add option `--server=${KUBERNETES_MASTER}` to kubectl in later steps. + +### Deploy etcd + +Start etcd and verify that it is running: + +{% highlight bash %} +sudo docker run -d --hostname $(uname -n) --name etcd \ + -p 4001:4001 -p 7001:7001 quay.io/coreos/etcd:v2.0.12 \ + --listen-client-urls http://0.0.0.0:4001 \ + --advertise-client-urls http://${KUBERNETES_MASTER_IP}:4001 +{% endhighlight %} + +{% highlight console %} +$ sudo docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +fd7bac9e2301 quay.io/coreos/etcd:v2.0.12 "/etcd" 5s ago Up 3s 2379/tcp, 2380/... etcd +{% endhighlight %} + +It's also a good idea to ensure your etcd instance is reachable by testing it + +{% highlight bash %} +curl -L http://${KUBERNETES_MASTER_IP}:4001/v2/keys/ +{% endhighlight %} + +If connectivity is OK, you will see an output of the available keys in etcd (if any). + +### Start Kubernetes-Mesos Services + +Update your PATH to more easily run the Kubernetes-Mesos binaries: + +{% highlight bash %} +export PATH="$(pwd)/_output/local/go/bin:$PATH" +{% endhighlight %} + +Identify your Mesos master: depending on your Mesos installation this is either a `host:port` like `mesos-master:5050` or a ZooKeeper URL like `zk://zookeeper:2181/mesos`. +In order to let Kubernetes survive Mesos master changes, the ZooKeeper URL is recommended for production environments. + +{% highlight bash %} +export MESOS_MASTER= +{% endhighlight %} + +Create a cloud config file `mesos-cloud.conf` in the current directory with the following contents: + +{% highlight console %} +$ cat <mesos-cloud.conf +[mesos-cloud] + mesos-master = ${MESOS_MASTER} +EOF +{% endhighlight %} + +Now start the kubernetes-mesos API server, controller manager, and scheduler on the master node: + +{% highlight console %} +$ km apiserver \ + --address=${KUBERNETES_MASTER_IP} \ + --etcd-servers=http://${KUBERNETES_MASTER_IP}:4001 \ + --service-cluster-ip-range=10.10.10.0/24 \ + --port=8888 \ + --cloud-provider=mesos \ + --cloud-config=mesos-cloud.conf \ + --secure-port=0 \ + --v=1 >apiserver.log 2>&1 & + +$ km controller-manager \ + --master=${KUBERNETES_MASTER_IP}:8888 \ + --cloud-provider=mesos \ + --cloud-config=./mesos-cloud.conf \ + --v=1 >controller.log 2>&1 & + +$ km scheduler \ + --address=${KUBERNETES_MASTER_IP} \ + --mesos-master=${MESOS_MASTER} \ + --etcd-servers=http://${KUBERNETES_MASTER_IP}:4001 \ + --mesos-user=root \ + --api-servers=${KUBERNETES_MASTER_IP}:8888 \ + --cluster-dns=10.10.10.10 \ + --cluster-domain=cluster.local \ + --v=2 >scheduler.log 2>&1 & +{% endhighlight %} + +Disown your background jobs so that they'll stay running if you log out. + +{% highlight bash %} +disown -a +{% endhighlight %} + +#### Validate KM Services + +Add the appropriate binary folder to your `PATH` to access kubectl: + +{% highlight bash %} +export PATH=/platforms/linux/amd64:$PATH +{% endhighlight %} + +Interact with the kubernetes-mesos framework via `kubectl`: + +{% highlight console %} +$ kubectl get pods +NAME READY STATUS RESTARTS AGE +{% endhighlight %} + +{% highlight console %} +# NOTE: your service IPs will likely differ +$ kubectl get services +NAME LABELS SELECTOR IP(S) PORT(S) +k8sm-scheduler component=scheduler,provider=k8sm 10.10.10.113 10251/TCP +kubernetes component=apiserver,provider=kubernetes 10.10.10.1 443/TCP +{% endhighlight %} + +Lastly, look for Kubernetes in the Mesos web GUI by pointing your browser to +`http://`. Make sure you have an active VPN connection. +Go to the Frameworks tab, and look for an active framework named "Kubernetes". + +## Spin up a pod + +Write a JSON pod description to a local file: + +{% highlight bash %} +$ cat <nginx.yaml +{% endhighlight %} + +{% highlight yaml %} +apiVersion: v1 +kind: Pod +metadata: + name: nginx +spec: + containers: + - name: nginx + image: nginx + ports: + - containerPort: 80 +EOPOD +{% endhighlight %} + +Send the pod description to Kubernetes using the `kubectl` CLI: + +{% highlight console %} +$ kubectl create -f ./nginx.yaml +pods/nginx +{% endhighlight %} + +Wait a minute or two while `dockerd` downloads the image layers from the internet. +We can use the `kubectl` interface to monitor the status of our pod: + +{% highlight console %} +$ kubectl get pods +NAME READY STATUS RESTARTS AGE +nginx 1/1 Running 0 14s +{% endhighlight %} + +Verify that the pod task is running in the Mesos web GUI. Click on the +Kubernetes framework. The next screen should show the running Mesos task that +started the Kubernetes pod. + +## Launching kube-dns + +Kube-dns is an addon for Kubernetes which adds DNS-based service discovery to the cluster. For a detailed explanation see [DNS in Kubernetes][4]. + +The kube-dns addon runs as a pod inside the cluster. The pod consists of three co-located containers: + +- a local etcd instance +- the [skydns][11] DNS server +- the kube2sky process to glue skydns to the state of the Kubernetes cluster. + +The skydns container offers DNS service via port 53 to the cluster. The etcd communication works via local 127.0.0.1 communication + +We assume that kube-dns will use + +- the service IP `10.10.10.10` +- and the `cluster.local` domain. + +Note that we have passed these two values already as parameter to the apiserver above. + +A template for an replication controller spinning up the pod with the 3 containers can be found at [cluster/addons/dns/skydns-rc.yaml.in][11] in the repository. The following steps are necessary in order to get a valid replication controller yaml file: + +- replace `{{ pillar['dns_replicas'] }}` with `1` +- replace `{{ pillar['dns_domain'] }}` with `cluster.local.` +- add `--kube_master_url=${KUBERNETES_MASTER}` parameter to the kube2sky container command. + +In addition the service template at [cluster/addons/dns/skydns-svc.yaml.in][12] needs the following replacement: + +- `{{ pillar['dns_server'] }}` with `10.10.10.10`. + +To do this automatically: + +{% highlight bash %} +sed -e "s/{{ pillar\['dns_replicas'\] }}/1/g;"\ +"s,\(command = \"/kube2sky\"\),\\1\\"$'\n'" - --kube_master_url=${KUBERNETES_MASTER},;"\ +"s/{{ pillar\['dns_domain'\] }}/cluster.local/g" \ + cluster/addons/dns/skydns-rc.yaml.in > skydns-rc.yaml +sed -e "s/{{ pillar\['dns_server'\] }}/10.10.10.10/g" \ + cluster/addons/dns/skydns-svc.yaml.in > skydns-svc.yaml +{% endhighlight %} + +Now the kube-dns pod and service are ready to be launched: + +{% highlight bash %} +kubectl create -f ./skydns-rc.yaml +kubectl create -f ./skydns-svc.yaml +{% endhighlight %} + +Check with `kubectl get pods --namespace=kube-system` that 3/3 containers of the pods are eventually up and running. Note that the kube-dns pods run in the `kube-system` namespace, not in `default`. + +To check that the new DNS service in the cluster works, we start a busybox pod and use that to do a DNS lookup. First create the `busybox.yaml` pod spec: + +{% highlight bash %} +cat <busybox.yaml +{% endhighlight %} + +{% highlight yaml %} +apiVersion: v1 +kind: Pod +metadata: + name: busybox + namespace: default +spec: + containers: + - image: busybox + command: + - sleep + - "3600" + imagePullPolicy: IfNotPresent + name: busybox + restartPolicy: Always +EOF +{% endhighlight %} + +Then start the pod: + +{% highlight bash %} +kubectl create -f ./busybox.yaml +{% endhighlight %} + +When the pod is up and running, start a lookup for the Kubernetes master service, made available on 10.10.10.1 by default: + +{% highlight bash %} +kubectl exec busybox -- nslookup kubernetes +{% endhighlight %} + +If everything works fine, you will get this output: + +{% highlight console %} +Server: 10.10.10.10 +Address 1: 10.10.10.10 + +Name: kubernetes +Address 1: 10.10.10.1 +{% endhighlight %} + +## What next? + +Try out some of the standard [Kubernetes examples][9]. + +Read about Kubernetes on Mesos' architecture in the [contrib directory][13]. + +**NOTE:** Some examples require Kubernetes DNS to be installed on the cluster. +Future work will add instructions to this guide to enable support for Kubernetes DNS. + +**NOTE:** Please be aware that there are [known issues with the current Kubernetes-Mesos implementation][7]. + +[1]: http://mesosphere.com/docs/tutorials/run-hadoop-on-mesos-using-installer +[2]: http://mesosphere.com/docs/tutorials/run-spark-on-mesos +[3]: http://mesosphere.com/docs/tutorials/run-chronos-on-mesos +[4]: https://releases.k8s.io/release-1.1/cluster/addons/dns/README.md +[5]: http://open.mesosphere.com/getting-started/cloud/google/mesosphere/ +[6]: http://mesos.apache.org/ +[7]: https://releases.k8s.io/release-1.1/contrib/mesos/docs/issues.md +[8]: https://github.com/mesosphere/kubernetes-mesos/issues +[9]: ../../examples/ +[10]: http://open.mesosphere.com/getting-started/cloud/google/mesosphere/#vpn-setup +[11]: https://releases.k8s.io/release-1.1/cluster/addons/dns/skydns-rc.yaml.in +[12]: https://releases.k8s.io/release-1.1/cluster/addons/dns/skydns-svc.yaml.in +[13]: https://releases.k8s.io/release-1.1/contrib/mesos/README.md + + + diff --git a/v1.1/docs/getting-started-guides/ovirt.md b/v1.1/docs/getting-started-guides/ovirt.md index af4b33c526..e299c42dfa 100644 --- a/v1.1/docs/getting-started-guides/ovirt.md +++ b/v1.1/docs/getting-started-guides/ovirt.md @@ -1,9 +1,6 @@ --- title: "Getting started on oVirt" -section: guides --- -## Table of Contents - {% include pagetoc.html %} ## What is oVirt diff --git a/v1.1/docs/getting-started-guides/rackspace.md b/v1.1/docs/getting-started-guides/rackspace.md index 8039770f3f..012c85662f 100644 --- a/v1.1/docs/getting-started-guides/rackspace.md +++ b/v1.1/docs/getting-started-guides/rackspace.md @@ -1,6 +1,5 @@ --- title: "Getting started on Rackspace" -section: guides --- ## Introduction @@ -17,7 +16,7 @@ The current cluster design is inspired by: - [corekube](https://github.com/metral/corekube) - [Angus Lees](https://github.com/anguslees/kube-openstack) -## Table of Contents + {% include pagetoc.html %} @@ -25,7 +24,7 @@ The current cluster design is inspired by: 1. Python2.7 2. You need to have both `nova` and `swiftly` installed. It's recommended to use a python virtualenv to install these packages into. -3. Make sure you have the appropriate environment variables set to interact with the OpenStack APIs. See [Rackspace Documentation](http://docs.rackspace.com/servers/api/v2/cs-gettingstarted/content/section_gs_install_nova.html) for more details. +3. Make sure you have the appropriate environment variables set to interact with the OpenStack APIs. See [Rackspace Documentation](http://docs.rackspace.com/servers/api/v2/cs-gettingstarted/content/section_gs_install_nova) for more details. ## Provider: Rackspace diff --git a/v1.1/docs/getting-started-guides/rkt/README.md b/v1.1/docs/getting-started-guides/rkt/README.md index 9ccecd3067..c19609b7d9 100644 --- a/v1.1/docs/getting-started-guides/rkt/README.md +++ b/v1.1/docs/getting-started-guides/rkt/README.md @@ -1,10 +1,6 @@ --- title: "Run Kubernetes with rkt" --- - - -# Run Kubernetes with rkt - This document describes how to run Kubernetes using [rkt](https://github.com/coreos/rkt) as a container runtime. We still have [a bunch of work](http://issue.k8s.io/8262) to do to make the experience with rkt wonderful, please stay tuned! @@ -27,19 +23,19 @@ If you are using the [hack/local-up-cluster.sh](https://releases.k8s.io/release- set these flags: {% highlight console %} -{% raw %} + $ export CONTAINER_RUNTIME=rkt $ export RKT_PATH=$PATH_TO_RKT_BINARY $ export RKT_STAGE1_IMAGE=PATH=$PATH_TO_STAGE1_IMAGE -{% endraw %} + {% endhighlight %} Then we can launch the local cluster using the script: {% highlight console %} -{% raw %} + $ hack/local-up-cluster.sh -{% endraw %} + {% endhighlight %} ### CoreOS cluster on Google Compute Engine (GCE) @@ -47,28 +43,28 @@ $ hack/local-up-cluster.sh To use rkt as the container runtime for your CoreOS cluster on GCE, you need to specify the OS distribution, project, image: {% highlight console %} -{% raw %} + $ export KUBE_OS_DISTRIBUTION=coreos $ export KUBE_GCE_MINION_IMAGE= $ export KUBE_GCE_MINION_PROJECT=coreos-cloud $ export KUBE_CONTAINER_RUNTIME=rkt -{% endraw %} + {% endhighlight %} You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`: {% highlight console %} -{% raw %} + $ export KUBE_RKT_VERSION=0.8.0 -{% endraw %} + {% endhighlight %} Then you can launch the cluster by: {% highlight console %} -{% raw %} + $ kube-up.sh -{% endraw %} + {% endhighlight %} Note that we are still working on making all containerized the master components run smoothly in rkt. Before that we are not able to run the master node with rkt yet. @@ -78,35 +74,35 @@ Note that we are still working on making all containerized the master components To use rkt as the container runtime for your CoreOS cluster on AWS, you need to specify the provider and OS distribution: {% highlight console %} -{% raw %} + $ export KUBERNETES_PROVIDER=aws $ export KUBE_OS_DISTRIBUTION=coreos $ export KUBE_CONTAINER_RUNTIME=rkt -{% endraw %} + {% endhighlight %} You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`: {% highlight console %} -{% raw %} + $ export KUBE_RKT_VERSION=0.8.0 -{% endraw %} + {% endhighlight %} You can optionally choose the CoreOS channel by setting `COREOS_CHANNEL`: {% highlight console %} -{% raw %} + $ export COREOS_CHANNEL=stable -{% endraw %} + {% endhighlight %} Then you can launch the cluster by: {% highlight console %} -{% raw %} + $ kube-up.sh -{% endraw %} + {% endhighlight %} Note: CoreOS is not supported as the master using the automated launch @@ -114,7 +110,7 @@ scripts. The master node is always Ubuntu. ### Getting started with your cluster -See [a simple nginx example](../../../docs/user-guide/simple-nginx.html) to try out your new cluster. +See [a simple nginx example](/{{page.version}}/docs/user-guide/simple-nginx) to try out your new cluster. For more complete applications, please look in the [examples directory](../../../examples/). @@ -142,9 +138,9 @@ using `journalctl`: - Check the running state of the systemd service: {% highlight console %} -{% raw %} + $ sudo journalctl -u $SERVICE_FILE -{% endraw %} + {% endhighlight %} where `$SERVICE_FILE` is the name of the service file created for the pod, you can find it in the kubelet logs. @@ -152,15 +148,15 @@ where `$SERVICE_FILE` is the name of the service file created for the pod, you c ##### Check the log of the container in the pod: {% highlight console %} -{% raw %} + $ sudo journalctl -M rkt-$UUID -u $CONTAINER_NAME -{% endraw %} + {% endhighlight %} where `$UUID` is the rkt pod's UUID, which you can find via `rkt list --full`, and `$CONTAINER_NAME` is the container's name. ##### Check Kubernetes events, logs. -Besides above tricks, Kubernetes also provides us handy tools for debugging the pods. More information can be found [here](../../../docs/user-guide/application-troubleshooting.html) +Besides above tricks, Kubernetes also provides us handy tools for debugging the pods. More information can be found [here](/{{page.version}}/docs/user-guide/application-troubleshooting) diff --git a/v1.1/docs/getting-started-guides/rkt/index.md b/v1.1/docs/getting-started-guides/rkt/index.md index 9ccecd3067..c19609b7d9 100644 --- a/v1.1/docs/getting-started-guides/rkt/index.md +++ b/v1.1/docs/getting-started-guides/rkt/index.md @@ -1,10 +1,6 @@ --- title: "Run Kubernetes with rkt" --- - - -# Run Kubernetes with rkt - This document describes how to run Kubernetes using [rkt](https://github.com/coreos/rkt) as a container runtime. We still have [a bunch of work](http://issue.k8s.io/8262) to do to make the experience with rkt wonderful, please stay tuned! @@ -27,19 +23,19 @@ If you are using the [hack/local-up-cluster.sh](https://releases.k8s.io/release- set these flags: {% highlight console %} -{% raw %} + $ export CONTAINER_RUNTIME=rkt $ export RKT_PATH=$PATH_TO_RKT_BINARY $ export RKT_STAGE1_IMAGE=PATH=$PATH_TO_STAGE1_IMAGE -{% endraw %} + {% endhighlight %} Then we can launch the local cluster using the script: {% highlight console %} -{% raw %} + $ hack/local-up-cluster.sh -{% endraw %} + {% endhighlight %} ### CoreOS cluster on Google Compute Engine (GCE) @@ -47,28 +43,28 @@ $ hack/local-up-cluster.sh To use rkt as the container runtime for your CoreOS cluster on GCE, you need to specify the OS distribution, project, image: {% highlight console %} -{% raw %} + $ export KUBE_OS_DISTRIBUTION=coreos $ export KUBE_GCE_MINION_IMAGE= $ export KUBE_GCE_MINION_PROJECT=coreos-cloud $ export KUBE_CONTAINER_RUNTIME=rkt -{% endraw %} + {% endhighlight %} You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`: {% highlight console %} -{% raw %} + $ export KUBE_RKT_VERSION=0.8.0 -{% endraw %} + {% endhighlight %} Then you can launch the cluster by: {% highlight console %} -{% raw %} + $ kube-up.sh -{% endraw %} + {% endhighlight %} Note that we are still working on making all containerized the master components run smoothly in rkt. Before that we are not able to run the master node with rkt yet. @@ -78,35 +74,35 @@ Note that we are still working on making all containerized the master components To use rkt as the container runtime for your CoreOS cluster on AWS, you need to specify the provider and OS distribution: {% highlight console %} -{% raw %} + $ export KUBERNETES_PROVIDER=aws $ export KUBE_OS_DISTRIBUTION=coreos $ export KUBE_CONTAINER_RUNTIME=rkt -{% endraw %} + {% endhighlight %} You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`: {% highlight console %} -{% raw %} + $ export KUBE_RKT_VERSION=0.8.0 -{% endraw %} + {% endhighlight %} You can optionally choose the CoreOS channel by setting `COREOS_CHANNEL`: {% highlight console %} -{% raw %} + $ export COREOS_CHANNEL=stable -{% endraw %} + {% endhighlight %} Then you can launch the cluster by: {% highlight console %} -{% raw %} + $ kube-up.sh -{% endraw %} + {% endhighlight %} Note: CoreOS is not supported as the master using the automated launch @@ -114,7 +110,7 @@ scripts. The master node is always Ubuntu. ### Getting started with your cluster -See [a simple nginx example](../../../docs/user-guide/simple-nginx.html) to try out your new cluster. +See [a simple nginx example](/{{page.version}}/docs/user-guide/simple-nginx) to try out your new cluster. For more complete applications, please look in the [examples directory](../../../examples/). @@ -142,9 +138,9 @@ using `journalctl`: - Check the running state of the systemd service: {% highlight console %} -{% raw %} + $ sudo journalctl -u $SERVICE_FILE -{% endraw %} + {% endhighlight %} where `$SERVICE_FILE` is the name of the service file created for the pod, you can find it in the kubelet logs. @@ -152,15 +148,15 @@ where `$SERVICE_FILE` is the name of the service file created for the pod, you c ##### Check the log of the container in the pod: {% highlight console %} -{% raw %} + $ sudo journalctl -M rkt-$UUID -u $CONTAINER_NAME -{% endraw %} + {% endhighlight %} where `$UUID` is the rkt pod's UUID, which you can find via `rkt list --full`, and `$CONTAINER_NAME` is the container's name. ##### Check Kubernetes events, logs. -Besides above tricks, Kubernetes also provides us handy tools for debugging the pods. More information can be found [here](../../../docs/user-guide/application-troubleshooting.html) +Besides above tricks, Kubernetes also provides us handy tools for debugging the pods. More information can be found [here](/{{page.version}}/docs/user-guide/application-troubleshooting) diff --git a/v1.1/docs/getting-started-guides/scratch.md b/v1.1/docs/getting-started-guides/scratch.md index c8f98168ad..18d3deffbf 100644 --- a/v1.1/docs/getting-started-guides/scratch.md +++ b/v1.1/docs/getting-started-guides/scratch.md @@ -1,823 +1,806 @@ ---- -title: "Creating a Custom Cluster From Scratch" -section: guides ---- - -This guide is for people who want to craft a custom Kubernetes cluster. If you -can find an existing Getting Started Guide that meets your needs on [this -list](/{{ page.version }}/docs/getting-started-guides/README/), then we recommend using it, as you will be able to benefit -from the experience of others. However, if you have specific IaaS, networking, -configuration management, or operating system requirements not met by any of -those guides, then this guide will provide an outline of the steps you need to -take. Note that it requires considerably more effort than using one of the -pre-defined guides. - -This guide is also useful for those wanting to understand at a high level some of the -steps that existing cluster setup scripts are making. - -## Table of Contents - -{% include pagetoc.html %} - -## Designing and Preparing - -### Learning - - 1. You should be familiar with using Kubernetes already. We suggest you set - up a temporary cluster by following one of the other Getting Started Guides. - This will help you become familiar with the CLI ([kubectl](/{{ page.version }}/docs/user-guide/kubectl/kubectl)) and concepts ([pods](/{{ page.version }}/docs/user-guide/pods), [services](/{{ page.version }}/docs/user-guide/services), etc.) first. - 1. You should have `kubectl` installed on your desktop. This will happen as a side - effect of completing one of the other Getting Started Guides. If not, follow the instructions - [here](/{{ page.version }}/docs/user-guide/prereqs). - -### Cloud Provider - -Kubernetes has the concept of a Cloud Provider, which is a module which provides -an interface for managing TCP Load Balancers, Nodes (Instances) and Networking Routes. -The interface is defined in `pkg/cloudprovider/cloud.go`. It is possible to -create a custom cluster without implementing a cloud provider (for example if using -bare-metal), and not all parts of the interface need to be implemented, depending -on how flags are set on various components. - -### Nodes - -- You can use virtual or physical machines. -- While you can build a cluster with 1 machine, in order to run all the examples and tests you - need at least 4 nodes. -- Many Getting-started-guides make a distinction between the master node and regular nodes. This - is not strictly necessary. -- Nodes will need to run some version of Linux with the x86_64 architecture. It may be possible - to run on other OSes and Architectures, but this guide does not try to assist with that. -- Apiserver and etcd together are fine on a machine with 1 core and 1GB RAM for clusters with 10s of nodes. - Larger or more active clusters may benefit from more cores. -- Other nodes can have any reasonable amount of memory and any number of cores. They need not - have identical configurations. - -### Network - -Kubernetes has a distinctive [networking model](/{{ page.version }}/docs/admin/networking). - -Kubernetes allocates an IP address to each pod. When creating a cluster, you -need to allocate a block of IPs for Kubernetes to use as Pod IPs. The simplest -approach is to allocate a different block of IPs to each node in the cluster as -the node is added. A process in one pod should be able to communicate with -another pod using the IP of the second pod. This connectivity can be -accomplished in two ways: -- Configure network to route Pod IPs - - Harder to setup from scratch. - - Google Compute Engine ([GCE](gce)) and [AWS](aws) guides use this approach. - - Need to make the Pod IPs routable by programming routers, switches, etc. - - Can be configured external to Kubernetes, or can implement in the "Routes" interface of a Cloud Provider module. - - Generally highest performance. -- Create an Overlay network - - Easier to setup - - Traffic is encapsulated, so per-pod IPs are routable. - - Examples: - - [Flannel](https://github.com/coreos/flannel) - - [Weave](http://weave.works/) - - [Open vSwitch (OVS)](http://openvswitch.org/) - - Does not require "Routes" portion of Cloud Provider module. - - Reduced performance (exactly how much depends on your solution). - -You need to select an address range for the Pod IPs. -- Various approaches: - - GCE: each project has its own `10.0.0.0/8`. Carve off a `/16` for each - Kubernetes cluster from that space, which leaves room for several clusters. - Each node gets a further subdivision of this space. - - AWS: use one VPC for whole organization, carve off a chunk for each - cluster, or use different VPC for different clusters. - - IPv6 is not supported yet. -- Allocate one CIDR subnet for each node's PodIPs, or a single large CIDR - from which smaller CIDRs are automatically allocated to each node (if nodes - are dynamically added). - - You need max-pods-per-node * max-number-of-nodes IPs in total. A `/24` per - node supports 254 pods per machine and is a common choice. If IPs are - scarce, a `/26` (62 pods per machine) or even a `/27` (30 pods) may be sufficient. - - e.g. use `10.10.0.0/16` as the range for the cluster, with up to 256 nodes - using `10.10.0.0/24` through `10.10.255.0/24`, respectively. - - Need to make these routable or connect with overlay. - -Kubernetes also allocates an IP to each [service](/{{ page.version }}/docs/user-guide/services). However, -service IPs do not necessarily need to be routable. The kube-proxy takes care -of translating Service IPs to Pod IPs before traffic leaves the node. You do -need to Allocate a block of IPs for services. Call this -`SERVICE_CLUSTER_IP_RANGE`. For example, you could set -`SERVICE_CLUSTER_IP_RANGE="10.0.0.0/16"`, allowing 65534 distinct services to -be active at once. Note that you can grow the end of this range, but you -cannot move it without disrupting the services and pods that already use it. - -Also, you need to pick a static IP for master node. -- Call this `MASTER_IP`. -- Open any firewalls to allow access to the apiserver ports 80 and/or 443. -- Enable ipv4 forwarding sysctl, `net.ipv4.ip_forward = 1` - -### Cluster Naming - -You should pick a name for your cluster. Pick a short name for each cluster -which is unique from future cluster names. This will be used in several ways: - - by kubectl to distinguish between various clusters you have access to. You will probably want a - second one sometime later, such as for testing new Kubernetes releases, running in a different -region of the world, etc. - - Kubernetes clusters can create cloud provider resources (e.g. AWS ELBs) and different clusters - need to distinguish which resources each created. Call this `CLUSTERNAME`. - -### Software Binaries - -You will need binaries for: - - etcd - - A container runner, one of: - - docker - - rkt - - Kubernetes - - kubelet - - kube-proxy - - kube-apiserver - - kube-controller-manager - - kube-scheduler - -#### Downloading and Extracting Kubernetes Binaries - -A Kubernetes binary release includes all the Kubernetes binaries as well as the supported release of etcd. -You can use a Kubernetes binary release (recommended) or build your Kubernetes binaries following the instructions in the -[Developer Documentation](/{{ page.version }}/docs/devel/README). Only using a binary release is covered in this guide. - -Download the [latest binary release](https://github.com/kubernetes/kubernetes/releases/latest) and unzip it. -Then locate `./kubernetes/server/kubernetes-server-linux-amd64.tar.gz` and unzip *that*. -Then, within the second set of unzipped files, locate `./kubernetes/server/bin`, which contains -all the necessary binaries. - -#### Selecting Images - -You will run docker, kubelet, and kube-proxy outside of a container, the same way you would run any system daemon, so -you just need the bare binaries. For etcd, kube-apiserver, kube-controller-manager, and kube-scheduler, -we recommend that you run these as containers, so you need an image to be built. - -You have several choices for Kubernetes images: -- Use images hosted on Google Container Registry (GCR): - - e.g `gcr.io/google_containers/hyperkube:$TAG`, where `TAG` is the latest - release tag, which can be found on the [latest releases page](https://github.com/kubernetes/kubernetes/releases/latest). - - Ensure $TAG is the same tag as the release tag you are using for kubelet and kube-proxy. - - The [hyperkube](https://releases.k8s.io/release-1.1/cmd/hyperkube) binary is an all in one binary - - `hyperkube kubelet ...` runs the kublet, `hyperkube apiserver ...` runs an apiserver, etc. -- Build your own images. - - Useful if you are using a private registry. - - The release contains files such as `./kubernetes/server/bin/kube-apiserver.tar` which - can be converted into docker images using a command like - `docker load -i kube-apiserver.tar` - - You can verify if the image is loaded successfully with the right repository and tag using - command like `docker images` - -For etcd, you can: -- Use images hosted on Google Container Registry (GCR), such as `gcr.io/google_containers/etcd:2.0.12` -- Use images hosted on [Docker Hub](https://hub.docker.com/search/?q=etcd) or [Quay.io](https://quay.io/repository/coreos/etcd), such as `quay.io/coreos/etcd:v2.2.0` -- Use etcd binary included in your OS distro. -- Build your own image - - You can do: `cd kubernetes/cluster/images/etcd; make` - -We recommend that you use the etcd version which is provided in the Kubernetes binary distribution. The Kubernetes binaries in the release -were tested extensively with this version of etcd and not with any other version. -The recommended version number can also be found as the value of `ETCD_VERSION` in `kubernetes/cluster/images/etcd/Makefile`. - -The remainder of the document assumes that the image identifiers have been chosen and stored in corresponding env vars. Examples (replace with latest tags and appropriate registry): - - `HYPERKUBE_IMAGE==gcr.io/google_containers/hyperkube:$TAG` - - `ETCD_IMAGE=gcr.io/google_containers/etcd:$ETCD_VERSION` - -### Security Models - -There are two main options for security: -- Access the apiserver using HTTP. - - Use a firewall for security. - - This is easier to setup. -- Access the apiserver using HTTPS - - Use https with certs, and credentials for user. - - This is the recommended approach. - - Configuring certs can be tricky. - -If following the HTTPS approach, you will need to prepare certs and credentials. - -#### Preparing Certs - -You need to prepare several certs: -- The master needs a cert to act as an HTTPS server. -- The kubelets optionally need certs to identify themselves as clients of the master, and when - serving its own API over HTTPS. - -Unless you plan to have a real CA generate your certs, you will need to generate a root cert and use that to sign the master, kubelet, and kubectl certs. -- see function `create-certs` in `cluster/gce/util.sh` -- see also `cluster/saltbase/salt/generate-cert/make-ca-cert.sh` and - `cluster/saltbase/salt/generate-cert/make-cert.sh` - -You will end up with the following files (we will use these variables later on) -- `CA_CERT` - - put in on node where apiserver runs, in e.g. `/srv/kubernetes/ca.crt`. -- `MASTER_CERT` - - signed by CA_CERT - - put in on node where apiserver runs, in e.g. `/srv/kubernetes/server.crt` -- `MASTER_KEY ` - - put in on node where apiserver runs, in e.g. `/srv/kubernetes/server.key` -- `KUBELET_CERT` - - optional -- `KUBELET_KEY` - - optional - -#### Preparing Credentials - -The admin user (and any users) need: - - a token or a password to identify them. - - tokens are just long alphanumeric strings, e.g. 32 chars. See - - `TOKEN=$(dd if=/dev/urandom bs=128 count=1 2>/dev/null | base64 | tr -d "=+/" | dd bs=32 count=1 2>/dev/null)` - -Your tokens and passwords need to be stored in a file for the apiserver -to read. This guide uses `/var/lib/kube-apiserver/known_tokens.csv`. -The format for this file is described in the [authentication documentation](/{{ page.version }}/docs/admin/authentication). - -For distributing credentials to clients, the convention in Kubernetes is to put the credentials -into a [kubeconfig file](/{{ page.version }}/docs/user-guide/kubeconfig-file). - -The kubeconfig file for the administrator can be created as follows: - - If you have already used Kubernetes with a non-custom cluster (for example, used a Getting Started - Guide), you will already have a `$HOME/.kube/config` file. - - You need to add certs, keys, and the master IP to the kubeconfig file: - - If using the firewall-only security option, set the apiserver this way: - - `kubectl config set-cluster $CLUSTER_NAME --server=http://$MASTER_IP --insecure-skip-tls-verify=true` - - Otherwise, do this to set the apiserver ip, client certs, and user credentials. - - `kubectl config set-cluster $CLUSTER_NAME --certificate-authority=$CA_CERT --embed-certs=true --server=https://$MASTER_IP` - - `kubectl config set-credentials $USER --client-certificate=$CLI_CERT --client-key=$CLI_KEY --embed-certs=true --token=$TOKEN` - - Set your cluster as the default cluster to use: - - `kubectl config set-context $CONTEXT_NAME --cluster=$CLUSTER_NAME --user=$USER` - - `kubectl config use-context $CONTEXT_NAME` - -Next, make a kubeconfig file for the kubelets and kube-proxy. There are a couple of options for how -many distinct files to make: - 1. Use the same credential as the admin - - This is simplest to setup. - 1. One token and kubeconfig file for all kubelets, one for all kube-proxy, one for admin. - - This mirrors what is done on GCE today - 1. Different credentials for every kubelet, etc. - - We are working on this but all the pieces are not ready yet. - -You can make the files by copying the `$HOME/.kube/config`, by following the code -in `cluster/gce/configure-vm.sh` or by using the following template: - -{% highlight yaml %} -{% raw %} -apiVersion: v1 -kind: Config -users: -- name: kubelet - user: - token: ${KUBELET_TOKEN} -clusters: -- name: local - cluster: - certificate-authority-data: ${CA_CERT_BASE64_ENCODED} -contexts: -- context: - cluster: local - user: kubelet - name: service-account-context -current-context: service-account-context -{% endraw %} -{% endhighlight %} - -Put the kubeconfig(s) on every node. The examples later in this -guide assume that there are kubeconfigs in `/var/lib/kube-proxy/kubeconfig` and -`/var/lib/kubelet/kubeconfig`. - -## Configuring and Installing Base Software on Nodes - -This section discusses how to configure machines to be Kubernetes nodes. - -You should run three daemons on every node: - - docker or rkt - - kubelet - - kube-proxy - -You will also need to do assorted other configuration on top of a -base OS install. - -Tip: One possible starting point is to setup a cluster using an existing Getting -Started Guide. After getting a cluster running, you can then copy the init.d scripts or systemd unit files from that -cluster, and then modify them for use on your custom cluster. - -### Docker - -The minimum required Docker version will vary as the kubelet version changes. The newest stable release is a good choice. Kubelet will log a warning and refuse to start pods if the version is too old, so pick a version and try it. - -If you previously had Docker installed on a node without setting Kubernetes-specific -options, you may have a Docker-created bridge and iptables rules. You may want to remove these -as follows before proceeding to configure Docker for Kubernetes. - -{% highlight sh %} -{% raw %} -iptables -t nat -F -ifconfig docker0 down -brctl delbr docker0 -{% endraw %} -{% endhighlight %} - -The way you configure docker will depend in whether you have chosen the routable-vip or overlay-network approaches for your network. -Some suggested docker options: - - create your own bridge for the per-node CIDR ranges, call it cbr0, and set `--bridge=cbr0` option on docker. - - set `--iptables=false` so docker will not manipulate iptables for host-ports (too coarse on older docker versions, may be fixed in newer versions) -so that kube-proxy can manage iptables instead of docker. - - `--ip-masq=false` - - if you have setup PodIPs to be routable, then you want this false, otherwise, docker will - rewrite the PodIP source-address to a NodeIP. - - some environments (e.g. GCE) still need you to masquerade out-bound traffic when it leaves the cloud environment. This is very environment specific. - - if you are using an overlay network, consult those instructions. - - `--mtu=` - - may be required when using Flannel, because of the extra packet size due to udp encapsulation - - `--insecure-registry $CLUSTER_SUBNET` - - to connect to a private registry, if you set one up, without using SSL. - -You may want to increase the number of open files for docker: - - `DOCKER_NOFILE=1000000` - -Where this config goes depends on your node OS. For example, GCE's Debian-based distro uses `/etc/default/docker`. - -Ensure docker is working correctly on your system before proceeding with the rest of the -installation, by following examples given in the Docker documentation. - -### rkt - -[rkt](https://github.com/coreos/rkt) is an alternative to Docker. You only need to install one of Docker or rkt. -The minimum version required is [v0.5.6](https://github.com/coreos/rkt/releases/tag/v0.5.6). - -[systemd](http://www.freedesktop.org/wiki/Software/systemd/) is required on your node to run rkt. The -minimum version required to match rkt v0.5.6 is -[systemd 215](http://lists.freedesktop.org/archives/systemd-devel/2014-July/020903). - -[rkt metadata service](https://github.com/coreos/rkt/blob/master/Documentation/networking.md) is also required -for rkt networking support. You can start rkt metadata service by using command like -`sudo systemd-run rkt metadata-service` - -Then you need to configure your kubelet with flag: - - `--container-runtime=rkt` - -### kubelet - -All nodes should run kubelet. See [Selecting Binaries](#selecting-binaries). - -Arguments to consider: - - If following the HTTPS security approach: - - `--api-servers=https://$MASTER_IP` - - `--kubeconfig=/var/lib/kubelet/kubeconfig` - - Otherwise, if taking the firewall-based security approach - - `--api-servers=http://$MASTER_IP` - - `--config=/etc/kubernetes/manifests` - - `--cluster-dns=` to the address of the DNS server you will setup (see [Starting Addons](#starting-addons).) - - `--cluster-domain=` to the dns domain prefix to use for cluster DNS addresses. - - `--docker-root=` - - `--root-dir=` - - `--configure-cbr0=` (described above) - - `--register-node` (described in [Node](/{{ page.version }}/docs/admin/node) documentation.) - -### kube-proxy - -All nodes should run kube-proxy. (Running kube-proxy on a "master" node is not -strictly required, but being consistent is easier.) Obtain a binary as described for -kubelet. - -Arguments to consider: - - If following the HTTPS security approach: - - `--api-servers=https://$MASTER_IP` - - `--kubeconfig=/var/lib/kube-proxy/kubeconfig` - - Otherwise, if taking the firewall-based security approach - - `--api-servers=http://$MASTER_IP` - -### Networking - -Each node needs to be allocated its own CIDR range for pod networking. -Call this `NODE_X_POD_CIDR`. - -A bridge called `cbr0` needs to be created on each node. The bridge is explained -further in the [networking documentation](/{{ page.version }}/docs/admin/networking). The bridge itself -needs an address from `$NODE_X_POD_CIDR` - by convention the first IP. Call -this `NODE_X_BRIDGE_ADDR`. For example, if `NODE_X_POD_CIDR` is `10.0.0.0/16`, -then `NODE_X_BRIDGE_ADDR` is `10.0.0.1/16`. NOTE: this retains the `/16` suffix -because of how this is used later. - -- Recommended, automatic approach: - 1. Set `--configure-cbr0=true` option in kubelet init script and restart kubelet service. Kubelet will configure cbr0 automatically. - It will wait to do this until the node controller has set Node.Spec.PodCIDR. Since you have not setup apiserver and node controller - yet, the bridge will not be setup immediately. -- Alternate, manual approach: - 1. Set `--configure-cbr0=false` on kubelet and restart. - 1. Create a bridge - - e.g. `brctl addbr cbr0`. - 1. Set appropriate MTU - - `ip link set dev cbr0 mtu 1460` (NOTE: the actual value of MTU will depend on your network environment) - 1. Add the clusters network to the bridge (docker will go on other side of bridge). - - e.g. `ip addr add $NODE_X_BRIDGE_ADDR dev eth0` - 1. Turn it on - - e.g. `ip link set dev cbr0 up` - -If you have turned off Docker's IP masquerading to allow pods to talk to each -other, then you may need to do masquerading just for destination IPs outside -the cluster network. For example: - -{% highlight sh %} -{% raw %} -iptables -w -t nat -A POSTROUTING -o eth0 -j MASQUERADE \! -d ${CLUSTER_SUBNET} -{% endraw %} -{% endhighlight %} - -This will rewrite the source address from -the PodIP to the Node IP for traffic bound outside the cluster, and kernel -[connection tracking](http://www.iptables.info/en/connection-state) -will ensure that responses destined to the node still reach -the pod. - -NOTE: This is environment specific. Some environments will not need -any masquerading at all. Others, such as GCE, will not allow pod IPs to send -traffic to the internet, but have no problem with them inside your GCE Project. - -### Other - -- Enable auto-upgrades for your OS package manager, if desired. -- Configure log rotation for all node components (e.g. using [logrotate](http://linux.die.net/man/8/logrotate)). -- Setup liveness-monitoring (e.g. using [supervisord](http://supervisord.org/)). -- Setup volume plugin support (optional) - - Install any client binaries for optional volume types, such as `glusterfs-client` for GlusterFS - volumes. - -### Using Configuration Management - -The previous steps all involved "conventional" system administration techniques for setting up -machines. You may want to use a Configuration Management system to automate the node configuration -process. There are examples of [Saltstack](/{{ page.version }}/docs/admin/salt), Ansible, Juju, and CoreOS Cloud Config in the -various Getting Started Guides. - -## Bootstrapping the Cluster - -While the basic node services (kubelet, kube-proxy, docker) are typically started and managed using -traditional system administration/automation approaches, the remaining *master* components of Kubernetes are -all configured and managed *by Kubernetes*: - - their options are specified in a Pod spec (yaml or json) rather than an /etc/init.d file or - systemd unit. - - they are kept running by Kubernetes rather than by init. - -### etcd - -You will need to run one or more instances of etcd. - - Recommended approach: run one etcd instance, with its log written to a directory backed - by durable storage (RAID, GCE PD) - - Alternative: run 3 or 5 etcd instances. - - Log can be written to non-durable storage because storage is replicated. - - run a single apiserver which connects to one of the etc nodes. - See [cluster-troubleshooting](/{{ page.version }}/docs/admin/cluster-troubleshooting) for more discussion on factors affecting cluster -availability. - -To run an etcd instance: - -1. copy `cluster/saltbase/salt/etcd/etcd.manifest` -1. make any modifications needed -1. start the pod by putting it into the kubelet manifest directory - -### Apiserver, Controller Manager, and Scheduler - -The apiserver, controller manager, and scheduler will each run as a pod on the master node. - -For each of these components, the steps to start them running are similar: - -1. Start with a provided template for a pod. -1. Set the `HYPERKUBE_IMAGE` to the values chosen in [Selecting Images](#selecting-images). -1. Determine which flags are needed for your cluster, using the advice below each template. -1. Set the flags to be individual strings in the command array (e.g. $ARGN below) -1. Start the pod by putting the completed template into the kubelet manifest directory. -1. Verify that the pod is started. - -#### Apiserver pod template - -{% highlight json %} -{% raw %} -{ - "kind": "Pod", - "apiVersion": "v1", - "metadata": { - "name": "kube-apiserver" - }, - "spec": { - "hostNetwork": true, - "containers": [ - { - "name": "kube-apiserver", - "image": "${HYPERKUBE_IMAGE}", - "command": [ - "/hyperkube", - "apiserver", - "$ARG1", - "$ARG2", - ... - "$ARGN" - ], - "ports": [ - { - "name": "https", - "hostPort": 443, - "containerPort": 443 - }, - { - "name": "local", - "hostPort": 8080, - "containerPort": 8080 - } - ], - "volumeMounts": [ - { - "name": "srvkube", - "mountPath": "/srv/kubernetes", - "readOnly": true - }, - { - "name": "etcssl", - "mountPath": "/etc/ssl", - "readOnly": true - } - ], - "livenessProbe": { - "httpGet": { - "path": "/healthz", - "port": 8080 - }, - "initialDelaySeconds": 15, - "timeoutSeconds": 15 - } - } - ], - "volumes": [ - { - "name": "srvkube", - "hostPath": { - "path": "/srv/kubernetes" - } - }, - { - "name": "etcssl", - "hostPath": { - "path": "/etc/ssl" - } - } - ] - } -} -{% endraw %} -{% endhighlight %} - -Here are some apiserver flags you may need to set: - -- `--cloud-provider=` see [cloud providers](#cloud-providers) -- `--cloud-config=` see [cloud providers](#cloud-providers) -- `--address=${MASTER_IP}` *or* `--bind-address=127.0.0.1` and `--address=127.0.0.1` if you want to run a proxy on the master node. -- `--cluster-name=$CLUSTER_NAME` -- `--service-cluster-ip-range=$SERVICE_CLUSTER_IP_RANGE` -- `--etcd-servers=http://127.0.0.1:4001` -- `--tls-cert-file=/srv/kubernetes/server.cert` -- `--tls-private-key-file=/srv/kubernetes/server.key` -- `--admission-control=$RECOMMENDED_LIST` - - See [admission controllers](/{{ page.version }}/docs/admin/admission-controllers) for recommended arguments. -- `--allow-privileged=true`, only if you trust your cluster user to run pods as root. - -If you are following the firewall-only security approach, then use these arguments: - -- `--token-auth-file=/dev/null` -- `--insecure-bind-address=$MASTER_IP` -- `--advertise-address=$MASTER_IP` - -If you are using the HTTPS approach, then set: -- `--client-ca-file=/srv/kubernetes/ca.crt` -- `--token-auth-file=/srv/kubernetes/known_tokens.csv` -- `--basic-auth-file=/srv/kubernetes/basic_auth.csv` - -This pod mounts several node file system directories using the `hostPath` volumes. Their purposes are: -- The `/etc/ssl` mount allows the apiserver to find the SSL root certs so it can - authenticate external services, such as a cloud provider. - - This is not required if you do not use a cloud provider (e.g. bare-metal). -- The `/srv/kubernetes` mount allows the apiserver to read certs and credentials stored on the - node disk. These could instead be stored on a persistent disk, such as a GCE PD, or baked into the image. -- Optionally, you may want to mount `/var/log` as well and redirect output there (not shown in template). - - Do this if you prefer your logs to be accessible from the root filesystem with tools like journalctl. - -*TODO* document proxy-ssh setup. - -##### Cloud Providers - -Apiserver supports several cloud providers. - -- options for `--cloud-provider` flag are `aws`, `gce`, `mesos`, `openshift`, `ovirt`, `rackspace`, `vagrant`, or unset. -- unset used for e.g. bare metal setups. -- support for new IaaS is added by contributing code [here](https://releases.k8s.io/release-1.1/pkg/cloudprovider/providers) - -Some cloud providers require a config file. If so, you need to put config file into apiserver image or mount through hostPath. - -- `--cloud-config=` set if cloud provider requires a config file. -- Used by `aws`, `gce`, `mesos`, `openshift`, `ovirt` and `rackspace`. -- You must put config file into apiserver image or mount through hostPath. -- Cloud config file syntax is [Gcfg](https://code.google.com/p/gcfg/). -- AWS format defined by type [AWSCloudConfig](https://releases.k8s.io/release-1.1/pkg/cloudprovider/providers/aws/aws.go) -- There is a similar type in the corresponding file for other cloud providers. -- GCE example: search for `gce.conf` in [this file](https://releases.k8s.io/release-1.1/cluster/gce/configure-vm.sh) - -#### Scheduler pod template - -Complete this template for the scheduler pod: - -{% highlight json %} -{% raw %} - -{ - "kind": "Pod", - "apiVersion": "v1", - "metadata": { - "name": "kube-scheduler" - }, - "spec": { - "hostNetwork": true, - "containers": [ - { - "name": "kube-scheduler", - "image": "$HYBERKUBE_IMAGE", - "command": [ - "/hyperkube", - "scheduler", - "--master=127.0.0.1:8080", - "$SCHEDULER_FLAG1", - ... - "$SCHEDULER_FLAGN" - ], - "livenessProbe": { - "httpGet": { - "host" : "127.0.0.1", - "path": "/healthz", - "port": 10251 - }, - "initialDelaySeconds": 15, - "timeoutSeconds": 15 - } - } - ] - } -} - -{% endraw %} -{% endhighlight %} - -Typically, no additional flags are required for the scheduler. - -Optionally, you may want to mount `/var/log` as well and redirect output there. - -#### Controller Manager Template - -Template for controller manager pod: - -{% highlight json %} -{% raw %} - -{ - "kind": "Pod", - "apiVersion": "v1", - "metadata": { - "name": "kube-controller-manager" - }, - "spec": { - "hostNetwork": true, - "containers": [ - { - "name": "kube-controller-manager", - "image": "$HYPERKUBE_IMAGE", - "command": [ - "/hyperkube", - "controller-manager", - "$CNTRLMNGR_FLAG1", - ... - "$CNTRLMNGR_FLAGN" - ], - "volumeMounts": [ - { - "name": "srvkube", - "mountPath": "/srv/kubernetes", - "readOnly": true - }, - { - "name": "etcssl", - "mountPath": "/etc/ssl", - "readOnly": true - } - ], - "livenessProbe": { - "httpGet": { - "host": "127.0.0.1", - "path": "/healthz", - "port": 10252 - }, - "initialDelaySeconds": 15, - "timeoutSeconds": 15 - } - } - ], - "volumes": [ - { - "name": "srvkube", - "hostPath": { - "path": "/srv/kubernetes" - } - }, - { - "name": "etcssl", - "hostPath": { - "path": "/etc/ssl" - } - } - ] - } -} - -{% endraw %} -{% endhighlight %} - -Flags to consider using with controller manager: - - `--cluster-name=$CLUSTER_NAME` - - `--cluster-cidr=` - - *TODO*: explain this flag. - - `--allocate-node-cidrs=` - - *TODO*: explain when you want controller to do this and when you want to do it another way. - - `--cloud-provider=` and `--cloud-config` as described in apiserver section. - - `--service-account-private-key-file=/srv/kubernetes/server.key`, used by the [service account](/{{ page.version }}/docs/user-guide/service-accounts) feature. - - `--master=127.0.0.1:8080` - -#### Starting and Verifying Apiserver, Scheduler, and Controller Manager - -Place each completed pod template into the kubelet config dir -(whatever `--config=` argument of kubelet is set to, typically -`/etc/kubernetes/manifests`). The order does not matter: scheduler and -controller manager will retry reaching the apiserver until it is up. - -Use `ps` or `docker ps` to verify that each process has started. For example, verify that kubelet has started a container for the apiserver like this: - -{% highlight console %} -{% raw %} -$ sudo docker ps | grep apiserver: -5783290746d5 gcr.io/google_containers/kube-apiserver:e36bf367342b5a80d7467fd7611ad873 "/bin/sh -c '/usr/lo'" 10 seconds ago Up 9 seconds k8s_kube-apiserver.feb145e7_kube-apiserver-kubernetes-master_default_eaebc600cf80dae59902b44225f2fc0a_225a4695 -{% endraw %} -{% endhighlight %} - -Then try to connect to the apiserver: - -{% highlight console %} -{% raw %} -$ echo $(curl -s http://localhost:8080/healthz) -ok -$ curl -s http://localhost:8080/api -{ - "versions": [ - "v1" - ] -} -{% endraw %} -{% endhighlight %} - -If you have selected the `--register-node=true` option for kubelets, they will now begin self-registering with the apiserver. -You should soon be able to see all your nodes by running the `kubectl get nodes` command. -Otherwise, you will need to manually create node objects. - -### Logging - -**TODO** talk about starting Logging. - -### Monitoring - -**TODO** talk about starting Monitoring. - -### DNS - -**TODO** talk about starting DNS. - -## Troubleshooting - -### Running validate-cluster - -**TODO** explain how to use `cluster/validate-cluster.sh` - -### Inspect pods and services - -Try to run through the "Inspect your cluster" section in one of the other Getting Started Guides, such as [GCE](/{{ page.version }}/docs/getting-started-guides/gce.html#inspect-your-cluster). -You should see some services. You should also see "mirror pods" for the apiserver, scheduler and controller-manager, plus any add-ons you started. - -### Try Examples - -At this point you should be able to run through one of the basic examples, such as the [nginx example](/{{ page.version }}/examples/simple-nginx). - -### Running the Conformance Test - -You may want to try to run the [Conformance test](http://releases.k8s.io/release-1.1/hack/conformance-test.sh). Any failures may give a hint as to areas that need more attention. - -### Networking - -The nodes must be able to connect to each other using their private IP. Verify this by -pinging or SSH-ing from one node to another. - -### Getting Help - -If you run into trouble, please see the section on [troubleshooting](/{{ page.version }}/docs/getting-started-guides/gce#troubleshooting), post to the -[google-containers group](https://groups.google.com/forum/#!forum/google-containers), or come ask questions on [Slack](/{{ page.version }}/docs/troubleshooting#slack). \ No newline at end of file +--- +title: "Creating a Custom Cluster From Scratch" +--- + +This guide is for people who want to craft a custom Kubernetes cluster. If you +can find an existing Getting Started Guide that meets your needs on [this +list](/{{page.version}}/docs/getting-started-guides/README/), then we recommend using it, as you will be able to benefit +from the experience of others. However, if you have specific IaaS, networking, +configuration management, or operating system requirements not met by any of +those guides, then this guide will provide an outline of the steps you need to +take. Note that it requires considerably more effort than using one of the +pre-defined guides. + +This guide is also useful for those wanting to understand at a high level some of the +steps that existing cluster setup scripts are making. + + + +{% include pagetoc.html %} + +## Designing and Preparing + +### Learning + + 1. You should be familiar with using Kubernetes already. We suggest you set + up a temporary cluster by following one of the other Getting Started Guides. + This will help you become familiar with the CLI ([kubectl](/{{page.version}}/docs/user-guide/kubectl/kubectl)) and concepts ([pods](/{{page.version}}/docs/user-guide/pods), [services](/{{page.version}}/docs/user-guide/services), etc.) first. + 1. You should have `kubectl` installed on your desktop. This will happen as a side + effect of completing one of the other Getting Started Guides. If not, follow the instructions + [here](/{{page.version}}/docs/user-guide/prereqs). + +### Cloud Provider + +Kubernetes has the concept of a Cloud Provider, which is a module which provides +an interface for managing TCP Load Balancers, Nodes (Instances) and Networking Routes. +The interface is defined in `pkg/cloudprovider/cloud.go`. It is possible to +create a custom cluster without implementing a cloud provider (for example if using +bare-metal), and not all parts of the interface need to be implemented, depending +on how flags are set on various components. + +### Nodes + +- You can use virtual or physical machines. +- While you can build a cluster with 1 machine, in order to run all the examples and tests you + need at least 4 nodes. +- Many Getting-started-guides make a distinction between the master node and regular nodes. This + is not strictly necessary. +- Nodes will need to run some version of Linux with the x86_64 architecture. It may be possible + to run on other OSes and Architectures, but this guide does not try to assist with that. +- Apiserver and etcd together are fine on a machine with 1 core and 1GB RAM for clusters with 10s of nodes. + Larger or more active clusters may benefit from more cores. +- Other nodes can have any reasonable amount of memory and any number of cores. They need not + have identical configurations. + +### Network + +Kubernetes has a distinctive [networking model](/{{page.version}}/docs/admin/networking). + +Kubernetes allocates an IP address to each pod. When creating a cluster, you +need to allocate a block of IPs for Kubernetes to use as Pod IPs. The simplest +approach is to allocate a different block of IPs to each node in the cluster as +the node is added. A process in one pod should be able to communicate with +another pod using the IP of the second pod. This connectivity can be +accomplished in two ways: +- Configure network to route Pod IPs + - Harder to setup from scratch. + - Google Compute Engine ([GCE](gce)) and [AWS](aws) guides use this approach. + - Need to make the Pod IPs routable by programming routers, switches, etc. + - Can be configured external to Kubernetes, or can implement in the "Routes" interface of a Cloud Provider module. + - Generally highest performance. +- Create an Overlay network + - Easier to setup + - Traffic is encapsulated, so per-pod IPs are routable. + - Examples: + - [Flannel](https://github.com/coreos/flannel) + - [Weave](http://weave.works/) + - [Open vSwitch (OVS)](http://openvswitch.org/) + - Does not require "Routes" portion of Cloud Provider module. + - Reduced performance (exactly how much depends on your solution). + +You need to select an address range for the Pod IPs. +- Various approaches: + - GCE: each project has its own `10.0.0.0/8`. Carve off a `/16` for each + Kubernetes cluster from that space, which leaves room for several clusters. + Each node gets a further subdivision of this space. + - AWS: use one VPC for whole organization, carve off a chunk for each + cluster, or use different VPC for different clusters. + - IPv6 is not supported yet. +- Allocate one CIDR subnet for each node's PodIPs, or a single large CIDR + from which smaller CIDRs are automatically allocated to each node (if nodes + are dynamically added). + - You need max-pods-per-node * max-number-of-nodes IPs in total. A `/24` per + node supports 254 pods per machine and is a common choice. If IPs are + scarce, a `/26` (62 pods per machine) or even a `/27` (30 pods) may be sufficient. + - e.g. use `10.10.0.0/16` as the range for the cluster, with up to 256 nodes + using `10.10.0.0/24` through `10.10.255.0/24`, respectively. + - Need to make these routable or connect with overlay. + +Kubernetes also allocates an IP to each [service](/{{page.version}}/docs/user-guide/services). However, +service IPs do not necessarily need to be routable. The kube-proxy takes care +of translating Service IPs to Pod IPs before traffic leaves the node. You do +need to Allocate a block of IPs for services. Call this +`SERVICE_CLUSTER_IP_RANGE`. For example, you could set +`SERVICE_CLUSTER_IP_RANGE="10.0.0.0/16"`, allowing 65534 distinct services to +be active at once. Note that you can grow the end of this range, but you +cannot move it without disrupting the services and pods that already use it. + +Also, you need to pick a static IP for master node. +- Call this `MASTER_IP`. +- Open any firewalls to allow access to the apiserver ports 80 and/or 443. +- Enable ipv4 forwarding sysctl, `net.ipv4.ip_forward = 1` + +### Cluster Naming + +You should pick a name for your cluster. Pick a short name for each cluster +which is unique from future cluster names. This will be used in several ways: + - by kubectl to distinguish between various clusters you have access to. You will probably want a + second one sometime later, such as for testing new Kubernetes releases, running in a different +region of the world, etc. + - Kubernetes clusters can create cloud provider resources (e.g. AWS ELBs) and different clusters + need to distinguish which resources each created. Call this `CLUSTERNAME`. + +### Software Binaries + +You will need binaries for: + - etcd + - A container runner, one of: + - docker + - rkt + - Kubernetes + - kubelet + - kube-proxy + - kube-apiserver + - kube-controller-manager + - kube-scheduler + +#### Downloading and Extracting Kubernetes Binaries + +A Kubernetes binary release includes all the Kubernetes binaries as well as the supported release of etcd. +You can use a Kubernetes binary release (recommended) or build your Kubernetes binaries following the instructions in the +[Developer Documentation](/{{page.version}}/docs/devel/README). Only using a binary release is covered in this guide. + +Download the [latest binary release](https://github.com/kubernetes/kubernetes/releases/latest) and unzip it. +Then locate `./kubernetes/server/kubernetes-server-linux-amd64.tar.gz` and unzip *that*. +Then, within the second set of unzipped files, locate `./kubernetes/server/bin`, which contains +all the necessary binaries. + +#### Selecting Images + +You will run docker, kubelet, and kube-proxy outside of a container, the same way you would run any system daemon, so +you just need the bare binaries. For etcd, kube-apiserver, kube-controller-manager, and kube-scheduler, +we recommend that you run these as containers, so you need an image to be built. + +You have several choices for Kubernetes images: +- Use images hosted on Google Container Registry (GCR): + - e.g `gcr.io/google_containers/hyperkube:$TAG`, where `TAG` is the latest + release tag, which can be found on the [latest releases page](https://github.com/kubernetes/kubernetes/releases/latest). + - Ensure $TAG is the same tag as the release tag you are using for kubelet and kube-proxy. + - The [hyperkube](https://releases.k8s.io/release-1.1/cmd/hyperkube) binary is an all in one binary + - `hyperkube kubelet ...` runs the kublet, `hyperkube apiserver ...` runs an apiserver, etc. +- Build your own images. + - Useful if you are using a private registry. + - The release contains files such as `./kubernetes/server/bin/kube-apiserver.tar` which + can be converted into docker images using a command like + `docker load -i kube-apiserver.tar` + - You can verify if the image is loaded successfully with the right repository and tag using + command like `docker images` + +For etcd, you can: +- Use images hosted on Google Container Registry (GCR), such as `gcr.io/google_containers/etcd:2.0.12` +- Use images hosted on [Docker Hub](https://hub.docker.com/search/?q=etcd) or [Quay.io](https://quay.io/repository/coreos/etcd), such as `quay.io/coreos/etcd:v2.2.0` +- Use etcd binary included in your OS distro. +- Build your own image + - You can do: `cd kubernetes/cluster/images/etcd; make` + +We recommend that you use the etcd version which is provided in the Kubernetes binary distribution. The Kubernetes binaries in the release +were tested extensively with this version of etcd and not with any other version. +The recommended version number can also be found as the value of `ETCD_VERSION` in `kubernetes/cluster/images/etcd/Makefile`. + +The remainder of the document assumes that the image identifiers have been chosen and stored in corresponding env vars. Examples (replace with latest tags and appropriate registry): + - `HYPERKUBE_IMAGE==gcr.io/google_containers/hyperkube:$TAG` + - `ETCD_IMAGE=gcr.io/google_containers/etcd:$ETCD_VERSION` + +### Security Models + +There are two main options for security: +- Access the apiserver using HTTP. + - Use a firewall for security. + - This is easier to setup. +- Access the apiserver using HTTPS + - Use https with certs, and credentials for user. + - This is the recommended approach. + - Configuring certs can be tricky. + +If following the HTTPS approach, you will need to prepare certs and credentials. + +#### Preparing Certs + +You need to prepare several certs: +- The master needs a cert to act as an HTTPS server. +- The kubelets optionally need certs to identify themselves as clients of the master, and when + serving its own API over HTTPS. + +Unless you plan to have a real CA generate your certs, you will need to generate a root cert and use that to sign the master, kubelet, and kubectl certs. +- see function `create-certs` in `cluster/gce/util.sh` +- see also `cluster/saltbase/salt/generate-cert/make-ca-cert.sh` and + `cluster/saltbase/salt/generate-cert/make-cert.sh` + +You will end up with the following files (we will use these variables later on) +- `CA_CERT` + - put in on node where apiserver runs, in e.g. `/srv/kubernetes/ca.crt`. +- `MASTER_CERT` + - signed by CA_CERT + - put in on node where apiserver runs, in e.g. `/srv/kubernetes/server.crt` +- `MASTER_KEY ` + - put in on node where apiserver runs, in e.g. `/srv/kubernetes/server.key` +- `KUBELET_CERT` + - optional +- `KUBELET_KEY` + - optional + +#### Preparing Credentials + +The admin user (and any users) need: + - a token or a password to identify them. + - tokens are just long alphanumeric strings, e.g. 32 chars. See + - `TOKEN=$(dd if=/dev/urandom bs=128 count=1 2>/dev/null | base64 | tr -d "=+/" | dd bs=32 count=1 2>/dev/null)` + +Your tokens and passwords need to be stored in a file for the apiserver +to read. This guide uses `/var/lib/kube-apiserver/known_tokens.csv`. +The format for this file is described in the [authentication documentation](/{{page.version}}/docs/admin/authentication). + +For distributing credentials to clients, the convention in Kubernetes is to put the credentials +into a [kubeconfig file](/{{page.version}}/docs/user-guide/kubeconfig-file). + +The kubeconfig file for the administrator can be created as follows: + - If you have already used Kubernetes with a non-custom cluster (for example, used a Getting Started + Guide), you will already have a `$HOME/.kube/config` file. + - You need to add certs, keys, and the master IP to the kubeconfig file: + - If using the firewall-only security option, set the apiserver this way: + - `kubectl config set-cluster $CLUSTER_NAME --server=http://$MASTER_IP --insecure-skip-tls-verify=true` + - Otherwise, do this to set the apiserver ip, client certs, and user credentials. + - `kubectl config set-cluster $CLUSTER_NAME --certificate-authority=$CA_CERT --embed-certs=true --server=https://$MASTER_IP` + - `kubectl config set-credentials $USER --client-certificate=$CLI_CERT --client-key=$CLI_KEY --embed-certs=true --token=$TOKEN` + - Set your cluster as the default cluster to use: + - `kubectl config set-context $CONTEXT_NAME --cluster=$CLUSTER_NAME --user=$USER` + - `kubectl config use-context $CONTEXT_NAME` + +Next, make a kubeconfig file for the kubelets and kube-proxy. There are a couple of options for how +many distinct files to make: + 1. Use the same credential as the admin + - This is simplest to setup. + 1. One token and kubeconfig file for all kubelets, one for all kube-proxy, one for admin. + - This mirrors what is done on GCE today + 1. Different credentials for every kubelet, etc. + - We are working on this but all the pieces are not ready yet. + +You can make the files by copying the `$HOME/.kube/config`, by following the code +in `cluster/gce/configure-vm.sh` or by using the following template: + +{% highlight yaml %} +apiVersion: v1 +kind: Config +users: +- name: kubelet + user: + token: ${KUBELET_TOKEN} +clusters: +- name: local + cluster: + certificate-authority-data: ${CA_CERT_BASE64_ENCODED} +contexts: +- context: + cluster: local + user: kubelet + name: service-account-context +current-context: service-account-context +{% endhighlight %} + +Put the kubeconfig(s) on every node. The examples later in this +guide assume that there are kubeconfigs in `/var/lib/kube-proxy/kubeconfig` and +`/var/lib/kubelet/kubeconfig`. + +## Configuring and Installing Base Software on Nodes + +This section discusses how to configure machines to be Kubernetes nodes. + +You should run three daemons on every node: + - docker or rkt + - kubelet + - kube-proxy + +You will also need to do assorted other configuration on top of a +base OS install. + +Tip: One possible starting point is to setup a cluster using an existing Getting +Started Guide. After getting a cluster running, you can then copy the init.d scripts or systemd unit files from that +cluster, and then modify them for use on your custom cluster. + +### Docker + +The minimum required Docker version will vary as the kubelet version changes. The newest stable release is a good choice. Kubelet will log a warning and refuse to start pods if the version is too old, so pick a version and try it. + +If you previously had Docker installed on a node without setting Kubernetes-specific +options, you may have a Docker-created bridge and iptables rules. You may want to remove these +as follows before proceeding to configure Docker for Kubernetes. + +{% highlight sh %} +iptables -t nat -F +ifconfig docker0 down +brctl delbr docker0 +{% endhighlight %} + +The way you configure docker will depend in whether you have chosen the routable-vip or overlay-network approaches for your network. +Some suggested docker options: + - create your own bridge for the per-node CIDR ranges, call it cbr0, and set `--bridge=cbr0` option on docker. + - set `--iptables=false` so docker will not manipulate iptables for host-ports (too coarse on older docker versions, may be fixed in newer versions) +so that kube-proxy can manage iptables instead of docker. + - `--ip-masq=false` + - if you have setup PodIPs to be routable, then you want this false, otherwise, docker will + rewrite the PodIP source-address to a NodeIP. + - some environments (e.g. GCE) still need you to masquerade out-bound traffic when it leaves the cloud environment. This is very environment specific. + - if you are using an overlay network, consult those instructions. + - `--mtu=` + - may be required when using Flannel, because of the extra packet size due to udp encapsulation + - `--insecure-registry $CLUSTER_SUBNET` + - to connect to a private registry, if you set one up, without using SSL. + +You may want to increase the number of open files for docker: + - `DOCKER_NOFILE=1000000` + +Where this config goes depends on your node OS. For example, GCE's Debian-based distro uses `/etc/default/docker`. + +Ensure docker is working correctly on your system before proceeding with the rest of the +installation, by following examples given in the Docker documentation. + +### rkt + +[rkt](https://github.com/coreos/rkt) is an alternative to Docker. You only need to install one of Docker or rkt. +The minimum version required is [v0.5.6](https://github.com/coreos/rkt/releases/tag/v0.5.6). + +[systemd](http://www.freedesktop.org/wiki/Software/systemd/) is required on your node to run rkt. The +minimum version required to match rkt v0.5.6 is +[systemd 215](http://lists.freedesktop.org/archives/systemd-devel/2014-July/020903). + +[rkt metadata service](https://github.com/coreos/rkt/blob/master/Documentation/networking.md) is also required +for rkt networking support. You can start rkt metadata service by using command like +`sudo systemd-run rkt metadata-service` + +Then you need to configure your kubelet with flag: + - `--container-runtime=rkt` + +### kubelet + +All nodes should run kubelet. See [Selecting Binaries](#selecting-binaries). + +Arguments to consider: + - If following the HTTPS security approach: + - `--api-servers=https://$MASTER_IP` + - `--kubeconfig=/var/lib/kubelet/kubeconfig` + - Otherwise, if taking the firewall-based security approach + - `--api-servers=http://$MASTER_IP` + - `--config=/etc/kubernetes/manifests` + - `--cluster-dns=` to the address of the DNS server you will setup (see [Starting Addons](#starting-addons).) + - `--cluster-domain=` to the dns domain prefix to use for cluster DNS addresses. + - `--docker-root=` + - `--root-dir=` + - `--configure-cbr0=` (described above) + - `--register-node` (described in [Node](/{{page.version}}/docs/admin/node) documentation.) + +### kube-proxy + +All nodes should run kube-proxy. (Running kube-proxy on a "master" node is not +strictly required, but being consistent is easier.) Obtain a binary as described for +kubelet. + +Arguments to consider: + - If following the HTTPS security approach: + - `--api-servers=https://$MASTER_IP` + - `--kubeconfig=/var/lib/kube-proxy/kubeconfig` + - Otherwise, if taking the firewall-based security approach + - `--api-servers=http://$MASTER_IP` + +### Networking + +Each node needs to be allocated its own CIDR range for pod networking. +Call this `NODE_X_POD_CIDR`. + +A bridge called `cbr0` needs to be created on each node. The bridge is explained +further in the [networking documentation](/{{page.version}}/docs/admin/networking). The bridge itself +needs an address from `$NODE_X_POD_CIDR` - by convention the first IP. Call +this `NODE_X_BRIDGE_ADDR`. For example, if `NODE_X_POD_CIDR` is `10.0.0.0/16`, +then `NODE_X_BRIDGE_ADDR` is `10.0.0.1/16`. NOTE: this retains the `/16` suffix +because of how this is used later. + +- Recommended, automatic approach: + 1. Set `--configure-cbr0=true` option in kubelet init script and restart kubelet service. Kubelet will configure cbr0 automatically. + It will wait to do this until the node controller has set Node.Spec.PodCIDR. Since you have not setup apiserver and node controller + yet, the bridge will not be setup immediately. +- Alternate, manual approach: + 1. Set `--configure-cbr0=false` on kubelet and restart. + 1. Create a bridge + - e.g. `brctl addbr cbr0`. + 1. Set appropriate MTU + - `ip link set dev cbr0 mtu 1460` (NOTE: the actual value of MTU will depend on your network environment) + 1. Add the clusters network to the bridge (docker will go on other side of bridge). + - e.g. `ip addr add $NODE_X_BRIDGE_ADDR dev eth0` + 1. Turn it on + - e.g. `ip link set dev cbr0 up` + +If you have turned off Docker's IP masquerading to allow pods to talk to each +other, then you may need to do masquerading just for destination IPs outside +the cluster network. For example: + +{% highlight sh %} +iptables -w -t nat -A POSTROUTING -o eth0 -j MASQUERADE \! -d ${CLUSTER_SUBNET} +{% endhighlight %} + +This will rewrite the source address from +the PodIP to the Node IP for traffic bound outside the cluster, and kernel +[connection tracking](http://www.iptables.info/en/connection-state) +will ensure that responses destined to the node still reach +the pod. + +NOTE: This is environment specific. Some environments will not need +any masquerading at all. Others, such as GCE, will not allow pod IPs to send +traffic to the internet, but have no problem with them inside your GCE Project. + +### Other + +- Enable auto-upgrades for your OS package manager, if desired. +- Configure log rotation for all node components (e.g. using [logrotate](http://linux.die.net/man/8/logrotate)). +- Setup liveness-monitoring (e.g. using [supervisord](http://supervisord.org/)). +- Setup volume plugin support (optional) + - Install any client binaries for optional volume types, such as `glusterfs-client` for GlusterFS + volumes. + +### Using Configuration Management + +The previous steps all involved "conventional" system administration techniques for setting up +machines. You may want to use a Configuration Management system to automate the node configuration +process. There are examples of [Saltstack](/{{page.version}}/docs/admin/salt), Ansible, Juju, and CoreOS Cloud Config in the +various Getting Started Guides. + +## Bootstrapping the Cluster + +While the basic node services (kubelet, kube-proxy, docker) are typically started and managed using +traditional system administration/automation approaches, the remaining *master* components of Kubernetes are +all configured and managed *by Kubernetes*: + - their options are specified in a Pod spec (yaml or json) rather than an /etc/init.d file or + systemd unit. + - they are kept running by Kubernetes rather than by init. + +### etcd + +You will need to run one or more instances of etcd. + - Recommended approach: run one etcd instance, with its log written to a directory backed + by durable storage (RAID, GCE PD) + - Alternative: run 3 or 5 etcd instances. + - Log can be written to non-durable storage because storage is replicated. + - run a single apiserver which connects to one of the etc nodes. + See [cluster-troubleshooting](/{{page.version}}/docs/admin/cluster-troubleshooting) for more discussion on factors affecting cluster +availability. + +To run an etcd instance: + +1. copy `cluster/saltbase/salt/etcd/etcd.manifest` +1. make any modifications needed +1. start the pod by putting it into the kubelet manifest directory + +### Apiserver, Controller Manager, and Scheduler + +The apiserver, controller manager, and scheduler will each run as a pod on the master node. + +For each of these components, the steps to start them running are similar: + +1. Start with a provided template for a pod. +1. Set the `HYPERKUBE_IMAGE` to the values chosen in [Selecting Images](#selecting-images). +1. Determine which flags are needed for your cluster, using the advice below each template. +1. Set the flags to be individual strings in the command array (e.g. $ARGN below) +1. Start the pod by putting the completed template into the kubelet manifest directory. +1. Verify that the pod is started. + +#### Apiserver pod template + +{% highlight json %} +{ + "kind": "Pod", + "apiVersion": "v1", + "metadata": { + "name": "kube-apiserver" + }, + "spec": { + "hostNetwork": true, + "containers": [ + { + "name": "kube-apiserver", + "image": "${HYPERKUBE_IMAGE}", + "command": [ + "/hyperkube", + "apiserver", + "$ARG1", + "$ARG2", + ... + "$ARGN" + ], + "ports": [ + { + "name": "https", + "hostPort": 443, + "containerPort": 443 + }, + { + "name": "local", + "hostPort": 8080, + "containerPort": 8080 + } + ], + "volumeMounts": [ + { + "name": "srvkube", + "mountPath": "/srv/kubernetes", + "readOnly": true + }, + { + "name": "etcssl", + "mountPath": "/etc/ssl", + "readOnly": true + } + ], + "livenessProbe": { + "httpGet": { + "path": "/healthz", + "port": 8080 + }, + "initialDelaySeconds": 15, + "timeoutSeconds": 15 + } + } + ], + "volumes": [ + { + "name": "srvkube", + "hostPath": { + "path": "/srv/kubernetes" + } + }, + { + "name": "etcssl", + "hostPath": { + "path": "/etc/ssl" + } + } + ] + } +} +{% endhighlight %} + +Here are some apiserver flags you may need to set: + +- `--cloud-provider=` see [cloud providers](#cloud-providers) +- `--cloud-config=` see [cloud providers](#cloud-providers) +- `--address=${MASTER_IP}` *or* `--bind-address=127.0.0.1` and `--address=127.0.0.1` if you want to run a proxy on the master node. +- `--cluster-name=$CLUSTER_NAME` +- `--service-cluster-ip-range=$SERVICE_CLUSTER_IP_RANGE` +- `--etcd-servers=http://127.0.0.1:4001` +- `--tls-cert-file=/srv/kubernetes/server.cert` +- `--tls-private-key-file=/srv/kubernetes/server.key` +- `--admission-control=$RECOMMENDED_LIST` + - See [admission controllers](/{{page.version}}/docs/admin/admission-controllers) for recommended arguments. +- `--allow-privileged=true`, only if you trust your cluster user to run pods as root. + +If you are following the firewall-only security approach, then use these arguments: + +- `--token-auth-file=/dev/null` +- `--insecure-bind-address=$MASTER_IP` +- `--advertise-address=$MASTER_IP` + +If you are using the HTTPS approach, then set: +- `--client-ca-file=/srv/kubernetes/ca.crt` +- `--token-auth-file=/srv/kubernetes/known_tokens.csv` +- `--basic-auth-file=/srv/kubernetes/basic_auth.csv` + +This pod mounts several node file system directories using the `hostPath` volumes. Their purposes are: +- The `/etc/ssl` mount allows the apiserver to find the SSL root certs so it can + authenticate external services, such as a cloud provider. + - This is not required if you do not use a cloud provider (e.g. bare-metal). +- The `/srv/kubernetes` mount allows the apiserver to read certs and credentials stored on the + node disk. These could instead be stored on a persistent disk, such as a GCE PD, or baked into the image. +- Optionally, you may want to mount `/var/log` as well and redirect output there (not shown in template). + - Do this if you prefer your logs to be accessible from the root filesystem with tools like journalctl. + +*TODO* document proxy-ssh setup. + +##### Cloud Providers + +Apiserver supports several cloud providers. + +- options for `--cloud-provider` flag are `aws`, `gce`, `mesos`, `openshift`, `ovirt`, `rackspace`, `vagrant`, or unset. +- unset used for e.g. bare metal setups. +- support for new IaaS is added by contributing code [here](https://releases.k8s.io/release-1.1/pkg/cloudprovider/providers) + +Some cloud providers require a config file. If so, you need to put config file into apiserver image or mount through hostPath. + +- `--cloud-config=` set if cloud provider requires a config file. +- Used by `aws`, `gce`, `mesos`, `openshift`, `ovirt` and `rackspace`. +- You must put config file into apiserver image or mount through hostPath. +- Cloud config file syntax is [Gcfg](https://code.google.com/p/gcfg/). +- AWS format defined by type [AWSCloudConfig](https://releases.k8s.io/release-1.1/pkg/cloudprovider/providers/aws/aws.go) +- There is a similar type in the corresponding file for other cloud providers. +- GCE example: search for `gce.conf` in [this file](https://releases.k8s.io/release-1.1/cluster/gce/configure-vm.sh) + +#### Scheduler pod template + +Complete this template for the scheduler pod: + +{% highlight json %} + +{ + "kind": "Pod", + "apiVersion": "v1", + "metadata": { + "name": "kube-scheduler" + }, + "spec": { + "hostNetwork": true, + "containers": [ + { + "name": "kube-scheduler", + "image": "$HYBERKUBE_IMAGE", + "command": [ + "/hyperkube", + "scheduler", + "--master=127.0.0.1:8080", + "$SCHEDULER_FLAG1", + ... + "$SCHEDULER_FLAGN" + ], + "livenessProbe": { + "httpGet": { + "host" : "127.0.0.1", + "path": "/healthz", + "port": 10251 + }, + "initialDelaySeconds": 15, + "timeoutSeconds": 15 + } + } + ] + } +} + +{% endhighlight %} + +Typically, no additional flags are required for the scheduler. + +Optionally, you may want to mount `/var/log` as well and redirect output there. + +#### Controller Manager Template + +Template for controller manager pod: + +{% highlight json %} + +{ + "kind": "Pod", + "apiVersion": "v1", + "metadata": { + "name": "kube-controller-manager" + }, + "spec": { + "hostNetwork": true, + "containers": [ + { + "name": "kube-controller-manager", + "image": "$HYPERKUBE_IMAGE", + "command": [ + "/hyperkube", + "controller-manager", + "$CNTRLMNGR_FLAG1", + ... + "$CNTRLMNGR_FLAGN" + ], + "volumeMounts": [ + { + "name": "srvkube", + "mountPath": "/srv/kubernetes", + "readOnly": true + }, + { + "name": "etcssl", + "mountPath": "/etc/ssl", + "readOnly": true + } + ], + "livenessProbe": { + "httpGet": { + "host": "127.0.0.1", + "path": "/healthz", + "port": 10252 + }, + "initialDelaySeconds": 15, + "timeoutSeconds": 15 + } + } + ], + "volumes": [ + { + "name": "srvkube", + "hostPath": { + "path": "/srv/kubernetes" + } + }, + { + "name": "etcssl", + "hostPath": { + "path": "/etc/ssl" + } + } + ] + } +} + +{% endhighlight %} + +Flags to consider using with controller manager: + - `--cluster-name=$CLUSTER_NAME` + - `--cluster-cidr=` + - *TODO*: explain this flag. + - `--allocate-node-cidrs=` + - *TODO*: explain when you want controller to do this and when you want to do it another way. + - `--cloud-provider=` and `--cloud-config` as described in apiserver section. + - `--service-account-private-key-file=/srv/kubernetes/server.key`, used by the [service account](/{{page.version}}/docs/user-guide/service-accounts) feature. + - `--master=127.0.0.1:8080` + +#### Starting and Verifying Apiserver, Scheduler, and Controller Manager + +Place each completed pod template into the kubelet config dir +(whatever `--config=` argument of kubelet is set to, typically +`/etc/kubernetes/manifests`). The order does not matter: scheduler and +controller manager will retry reaching the apiserver until it is up. + +Use `ps` or `docker ps` to verify that each process has started. For example, verify that kubelet has started a container for the apiserver like this: + +{% highlight console %} +$ sudo docker ps | grep apiserver: +5783290746d5 gcr.io/google_containers/kube-apiserver:e36bf367342b5a80d7467fd7611ad873 "/bin/sh -c '/usr/lo'" 10 seconds ago Up 9 seconds k8s_kube-apiserver.feb145e7_kube-apiserver-kubernetes-master_default_eaebc600cf80dae59902b44225f2fc0a_225a4695 +{% endhighlight %} + +Then try to connect to the apiserver: + +{% highlight console %} +$ echo $(curl -s http://localhost:8080/healthz) +ok +$ curl -s http://localhost:8080/api +{ + "versions": [ + "v1" + ] +} +{% endhighlight %} + +If you have selected the `--register-node=true` option for kubelets, they will now begin self-registering with the apiserver. +You should soon be able to see all your nodes by running the `kubectl get nodes` command. +Otherwise, you will need to manually create node objects. + +### Logging + +**TODO** talk about starting Logging. + +### Monitoring + +**TODO** talk about starting Monitoring. + +### DNS + +**TODO** talk about starting DNS. + +## Troubleshooting + +### Running validate-cluster + +**TODO** explain how to use `cluster/validate-cluster.sh` + +### Inspect pods and services + +Try to run through the "Inspect your cluster" section in one of the other Getting Started Guides, such as [GCE](/{{page.version}}/docs/getting-started-guides/gce.html#inspect-your-cluster). +You should see some services. You should also see "mirror pods" for the apiserver, scheduler and controller-manager, plus any add-ons you started. + +### Try Examples + +At this point you should be able to run through one of the basic examples, such as the [nginx example](/{{page.version}}/examples/simple-nginx). + +### Running the Conformance Test + +You may want to try to run the [Conformance test](http://releases.k8s.io/release-1.1/hack/conformance-test.sh). Any failures may give a hint as to areas that need more attention. + +### Networking + +The nodes must be able to connect to each other using their private IP. Verify this by +pinging or SSH-ing from one node to another. + +### Getting Help + +If you run into trouble, please see the section on [troubleshooting](/{{page.version}}/docs/getting-started-guides/gce#troubleshooting), post to the +[google-containers group](https://groups.google.com/forum/#!forum/google-containers), or come ask questions on [Slack](/{{page.version}}/docs/troubleshooting#slack). \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/ubuntu-calico.md b/v1.1/docs/getting-started-guides/ubuntu-calico.md index 1fd2a09bbb..2f626852c7 100644 --- a/v1.1/docs/getting-started-guides/ubuntu-calico.md +++ b/v1.1/docs/getting-started-guides/ubuntu-calico.md @@ -1,10 +1,6 @@ --- title: "Kubernetes Deployment On Bare-metal Ubuntu Nodes with Calico Networking" --- - -Kubernetes Deployment On Bare-metal Ubuntu Nodes with Calico Networking ------------------------------------------------- - ## Introduction This document describes how to deploy Kubernetes on Ubuntu bare metal nodes with Calico Networking plugin. See [projectcalico.org](http://projectcalico.org) for more information on what Calico is, and [the calicoctl github](https://github.com/projectcalico/calico-docker) for more information on the command-line tool, `calicoctl`. @@ -36,10 +32,10 @@ On each Node: First, get the sample configurations for this tutorial ``` -{% raw %} + wget https://github.com/Metaswitch/calico-kubernetes-ubuntu-demo/archive/master.tar.gz tar -xvf master.tar.gz -{% endraw %} + ``` ### Setup environment variables for systemd services on Master @@ -49,9 +45,9 @@ Many of the sample systemd services provided rely on environment variables on a 1.) Copy the network-environment-template from the `master` directory for editing. ``` -{% raw %} + cp calico-kubernetes-ubuntu-demo-master/master/network-environment-template network-environment -{% endraw %} + ``` 2.) Edit `network-environment` to represent your current host's settings. @@ -59,9 +55,9 @@ cp calico-kubernetes-ubuntu-demo-master/master/network-environment-template netw 3.) Move the `network-environment` into `/etc` ``` -{% raw %} + sudo mv -f network-environment /etc -{% endraw %} + ``` ### Install Kubernetes on Master @@ -69,7 +65,7 @@ sudo mv -f network-environment /etc 1.) Build & Install Kubernetes binaries ``` -{% raw %} + # Get the Kubernetes Source wget https://github.com/kubernetes/kubernetes/releases/download/v1.0.3/kubernetes.tar.gz @@ -81,30 +77,30 @@ kubernetes/cluster/ubuntu/build.sh # Add binaries to /usr/bin sudo cp -f binaries/master/* /usr/bin sudo cp -f binaries/kubectl /usr/bin -{% endraw %} + ``` 2.) Install the sample systemd processes settings for launching kubernetes services ``` -{% raw %} + sudo cp -f calico-kubernetes-ubuntu-demo-master/master/*.service /etc/systemd sudo systemctl enable /etc/systemd/etcd.service sudo systemctl enable /etc/systemd/kube-apiserver.service sudo systemctl enable /etc/systemd/kube-controller-manager.service sudo systemctl enable /etc/systemd/kube-scheduler.service -{% endraw %} + ``` 3.) Launch the processes. ``` -{% raw %} + sudo systemctl start etcd.service sudo systemctl start kube-apiserver.service sudo systemctl start kube-controller-manager.service sudo systemctl start kube-scheduler.service -{% endraw %} + ``` ### Install Calico on Master @@ -112,7 +108,7 @@ sudo systemctl start kube-scheduler.service In order to allow the master to route to pods on our nodes, we will launch the calico-node daemon on our master. This will allow it to learn routes over BGP from the other calico-node daemons in the cluster. The docker daemon should already be running before calico is started. ``` -{% raw %} + # Install the calicoctl binary, which will be used to launch calico wget https://github.com/projectcalico/calico-docker/releases/download/v0.5.5/calicoctl chmod +x calicoctl @@ -122,7 +118,7 @@ sudo cp -f calicoctl /usr/bin sudo cp -f calico-kubernetes-ubuntu-demo-master/master/calico-node.service /etc/systemd sudo systemctl enable /etc/systemd/calico-node.service sudo systemctl start calico-node.service -{% endraw %} + ``` >Note: calico-node may take a few minutes on first boot while it downloads the calico-node docker image. @@ -136,18 +132,18 @@ Perform these steps **once on each node**, ensuring you appropriately set the en 1.) Get the sample configurations for this tutorial ``` -{% raw %} + wget https://github.com/Metaswitch/calico-kubernetes-ubuntu-demo/archive/master.tar.gz tar -xvf master.tar.gz -{% endraw %} + ``` 2.) Copy the network-environment-template from the `node` directory ``` -{% raw %} + cp calico-kubernetes-ubuntu-demo-master/node/network-environment-template network-environment -{% endraw %} + ``` 3.) Edit `network-environment` to represent your current host's settings. @@ -155,9 +151,9 @@ cp calico-kubernetes-ubuntu-demo-master/node/network-environment-template networ 4.) Move `network-environment` into `/etc` ``` -{% raw %} + sudo mv -f network-environment /etc -{% endraw %} + ``` ### Configure Docker on the Node @@ -167,12 +163,12 @@ sudo mv -f network-environment /etc Instead of using docker's default interface (docker0), we will configure a new one to use desired IP ranges ``` -{% raw %} + sudo apt-get install -y bridge-utils sudo brctl addbr cbr0 sudo ifconfig cbr0 up sudo ifconfig cbr0 /24 -{% endraw %} + ``` > Replace \ with the subnet for this host's containers. Example topology: @@ -194,10 +190,10 @@ The Docker daemon must be started and told to use the already configured cbr0 in 3.) Reload systemctl and restart docker. ``` -{% raw %} + sudo systemctl daemon-reload sudo systemctl restart docker -{% endraw %} + ``` ### Install Calico on the Node @@ -205,7 +201,7 @@ sudo systemctl restart docker 1.) Install Calico ``` -{% raw %} + # Get the calicoctl binary wget https://github.com/projectcalico/calico-docker/releases/download/v0.5.5/calicoctl chmod +x calicoctl @@ -215,7 +211,7 @@ sudo cp -f calicoctl /usr/bin sudo cp calico-kubernetes-ubuntu-demo-master/node/calico-node.service /etc/systemd sudo systemctl enable /etc/systemd/calico-node.service sudo systemctl start calico-node.service -{% endraw %} + ``` >The calico-node service will automatically get the kubernetes-calico plugin binary and install it on the host system. @@ -224,9 +220,9 @@ sudo systemctl start calico-node.service **NOTE: This step only needs to be performed once per Kubernetes deployment, as it covers all the node's IP ranges.** ``` -{% raw %} + ETCD_AUTHORITY=:4001 calicoctl pool add 192.168.0.0/16 -{% endraw %} + ``` ### Install Kubernetes on the Node @@ -234,7 +230,7 @@ ETCD_AUTHORITY=:4001 calicoctl pool add 192.168.0.0/16 1.) Build & Install Kubernetes binaries ``` -{% raw %} + # Get the Kubernetes Source wget https://github.com/kubernetes/kubernetes/releases/download/v1.0.3/kubernetes.tar.gz @@ -250,20 +246,20 @@ sudo cp -f binaries/minion/* /usr/bin wget https://github.com/projectcalico/calico-kubernetes/releases/download/v0.1.1/kube-proxy sudo cp kube-proxy /usr/bin/ sudo chmod +x /usr/bin/kube-proxy -{% endraw %} + ``` 2.) Install and launch the sample systemd processes settings for launching Kubernetes services ``` -{% raw %} + sudo cp calico-kubernetes-ubuntu-demo-master/node/kube-proxy.service /etc/systemd/ sudo cp calico-kubernetes-ubuntu-demo-master/node/kube-kubelet.service /etc/systemd/ sudo systemctl enable /etc/systemd/kube-proxy.service sudo systemctl enable /etc/systemd/kube-kubelet.service sudo systemctl start kube-proxy.service sudo systemctl start kube-kubelet.service -{% endraw %} + ``` >*You may want to consider checking their status after to ensure everything is running* @@ -286,7 +282,7 @@ With this sample configuration, because the containers have private `192.168.0.0 ### NAT on the nodes -The simplest method for enabling connectivity from containers to the internet is to use an iptables masquerade rule. This is the standard mechanism [recommended](../../docs/admin/networking.html#google-compute-engine-gce) in the Kubernetes GCE environment. +The simplest method for enabling connectivity from containers to the internet is to use an iptables masquerade rule. This is the standard mechanism [recommended](/{{page.version}}/docs/admin/networking.html#google-compute-engine-gce) in the Kubernetes GCE environment. We need to NAT traffic that has a destination outside of the cluster. Internal traffic includes the master/nodes, and the container IP pools. A suitable masquerade chain would follow the pattern below, replacing the following variables: - `CONTAINER_SUBNET`: The cluster-wide subnet from which container IPs are chosen. All cbr0 bridge subnets fall within this range. The above example uses `192.168.0.0/16`. @@ -294,13 +290,13 @@ We need to NAT traffic that has a destination outside of the cluster. Internal t - `HOST_INTERFACE`: The interface on the Kubernetes node which is used for external connectivity. The above example uses `eth0` ``` -{% raw %} + sudo iptables -t nat -N KUBE-OUTBOUND-NAT sudo iptables -t nat -A KUBE-OUTBOUND-NAT -d -o -j RETURN sudo iptables -t nat -A KUBE-OUTBOUND-NAT -d -o -j RETURN sudo iptables -t nat -A KUBE-OUTBOUND-NAT -j MASQUERADE sudo iptables -t nat -A POSTROUTING -j KUBE-OUTBOUND-NAT -{% endraw %} + ``` This chain should be applied on the master and all nodes. In production, these rules should be persisted, e.g. with `iptables-persistent`. diff --git a/v1.1/docs/getting-started-guides/ubuntu.md b/v1.1/docs/getting-started-guides/ubuntu.md index c41443cf21..b61e2cfb5a 100644 --- a/v1.1/docs/getting-started-guides/ubuntu.md +++ b/v1.1/docs/getting-started-guides/ubuntu.md @@ -1,6 +1,5 @@ --- title: "Kubernetes Deployment On Bare-metal Ubuntu Nodes" -section: guides --- ## Introduction @@ -12,7 +11,6 @@ work, which has been merge into this document. [Cloud team from Zhejiang University](https://github.com/ZJU-SEL) will maintain this work. -## Table of Contents {% include pagetoc.html %} @@ -33,18 +31,18 @@ Ubuntu 15 which use systemd instead of upstart. We are working around fixing thi First clone the kubernetes github repo {% highlight console %} -{% raw %} + $ git clone https://github.com/kubernetes/kubernetes.git -{% endraw %} + {% endhighlight %} Then download all the needed binaries into given directory (cluster/ubuntu/binaries) {% highlight console %} -{% raw %} + $ cd kubernetes/cluster/ubuntu $ ./build.sh -{% endraw %} + {% endhighlight %} You can customize your etcd version, flannel version, k8s version by changing corresponding variables @@ -70,7 +68,7 @@ An example cluster is listed below: First configure the cluster information in cluster/ubuntu/config-default.sh, below is a simple sample. {% highlight sh %} -{% raw %} + export nodes="vcap@10.10.103.250 vcap@10.10.103.162 vcap@10.10.103.223" export role="ai i i" @@ -80,7 +78,7 @@ export NUM_MINIONS=${NUM_MINIONS:-3} export SERVICE_CLUSTER_IP_RANGE=192.168.3.0/24 export FLANNEL_NET=172.16.0.0/16 -{% endraw %} + {% endhighlight %} The first variable `nodes` defines all your cluster nodes, MASTER node comes first and @@ -117,19 +115,19 @@ The scripts automatically scp binaries and config files to all the machines and The only thing you need to do is to type the sudo password when promoted. {% highlight console %} -{% raw %} + Deploying minion on machine 10.10.103.223 ... [sudo] password to copy files and start minion: -{% endraw %} + {% endhighlight %} If all things goes right, you will see the below message from console indicating the k8s is up. {% highlight console %} -{% raw %} + Cluster validation succeeded -{% endraw %} + {% endhighlight %} ### Test it out @@ -141,13 +139,13 @@ You can make it available via PATH, then you can use the below command smoothly. For example, use `$ kubectl get nodes` to see if all of your nodes are ready. {% highlight console %} -{% raw %} + $ kubectl get nodes NAME LABELS STATUS 10.10.103.162 kubernetes.io/hostname=10.10.103.162 Ready 10.10.103.223 kubernetes.io/hostname=10.10.103.223 Ready 10.10.103.250 kubernetes.io/hostname=10.10.103.250 Ready -{% endraw %} + {% endhighlight %} Also you can run Kubernetes [guest-example](../../examples/guestbook/) to build a redis backend cluster on the k8s． @@ -161,7 +159,7 @@ and UI onto the existing cluster. The configuration of DNS is configured in cluster/ubuntu/config-default.sh. {% highlight sh %} -{% raw %} + ENABLE_CLUSTER_DNS="${KUBE_ENABLE_CLUSTER_DNS:-true}" DNS_SERVER_IP="192.168.3.10" @@ -169,7 +167,7 @@ DNS_SERVER_IP="192.168.3.10" DNS_DOMAIN="cluster.local" DNS_REPLICAS=1 -{% endraw %} + {% endhighlight %} The `DNS_SERVER_IP` is defining the ip of dns server which must be in the `SERVICE_CLUSTER_IP_RANGE`. @@ -178,18 +176,18 @@ The `DNS_REPLICAS` describes how many dns pod running in the cluster. By default, we also take care of kube-ui addon. {% highlight sh %} -{% raw %} + ENABLE_CLUSTER_UI="${KUBE_ENABLE_CLUSTER_UI:-true}" -{% endraw %} + {% endhighlight %} After all the above variables have been set, just type the following command. {% highlight console %} -{% raw %} + $ cd cluster/ubuntu $ KUBERNETES_PROVIDER=ubuntu ./deployAddons.sh -{% endraw %} + {% endhighlight %} After some time, you can use `$ kubectl get pods --namespace=kube-system` to see the DNS and UI pods are running in the cluster. @@ -225,10 +223,10 @@ Please try: the latter one could start it again. {% highlight console %} -{% raw %} + $ KUBERNETES_PROVIDER=ubuntu ./kube-down.sh $ KUBERNETES_PROVIDER=ubuntu ./kube-up.sh -{% endraw %} + {% endhighlight %} 4. You can also customize your own settings in `/etc/default/{component_name}`. @@ -240,39 +238,39 @@ If you already have a kubernetes cluster, and want to upgrade to a new version, you can use following command in cluster/ directory to update the whole cluster or a specified node to a new version. {% highlight console %} -{% raw %} + $ KUBERNETES_PROVIDER=ubuntu ./kube-push.sh [-m|-n ] -{% endraw %} + {% endhighlight %} It can be done for all components (by default), master(`-m`) or specified node(`-n`). If the version is not specified, the script will try to use local binaries.You should ensure all the binaries are well prepared in path `cluster/ubuntu/binaries`. {% highlight console %} -{% raw %} + $ tree cluster/ubuntu/binaries binaries/ -├── kubectl -├── master -│   ├── etcd -│   ├── etcdctl -│   ├── flanneld -│   ├── kube-apiserver -│   ├── kube-controller-manager -│   └── kube-scheduler -└── minion - ├── flanneld - ├── kubelet - └── kube-proxy -{% endraw %} +'��'��'�� kubectl +'��'��'�� master +'��   '��'��'�� etcd +'��   '��'��'�� etcdctl +'��   '��'��'�� flanneld +'��   '��'��'�� kube-apiserver +'��   '��'��'�� kube-controller-manager +'��   '��'��'�� kube-scheduler +'��'��'�� minion + '��'��'�� flanneld + '��'��'�� kubelet + '��'��'�� kube-proxy + {% endhighlight %} Upgrading single node is experimental now. You can use following command to get a help. {% highlight console %} -{% raw %} + $ KUBERNETES_PROVIDER=ubuntu ./kube-push.sh -h -{% endraw %} + {% endhighlight %} Some examples are as follows: diff --git a/v1.1/docs/getting-started-guides/vagrant.md b/v1.1/docs/getting-started-guides/vagrant.md index 0400afcff4..4259d249a2 100644 --- a/v1.1/docs/getting-started-guides/vagrant.md +++ b/v1.1/docs/getting-started-guides/vagrant.md @@ -1,399 +1,338 @@ ---- -title: "Getting started with Vagrant" -section: guides ---- - -Running Kubernetes with Vagrant (and VirtualBox) is an easy way to run/test/develop on your local machine (Linux, Mac OS X). - -## Table of Contents - -{% include pagetoc.html %} - -### Prerequisites - -1. Install latest version >= 1.6.2 of vagrant from http://www.vagrantup.com/downloads.html -2. Install one of: - 1. Version 4.3.28 of Virtual Box from https://www.virtualbox.org/wiki/Download_Old_Builds_4_3 - 2. [VMWare Fusion](https://www.vmware.com/products/fusion/) version 5 or greater as well as the appropriate [Vagrant VMWare Fusion provider](https://www.vagrantup.com/vmware) - 3. [VMWare Workstation](https://www.vmware.com/products/workstation/) version 9 or greater as well as the [Vagrant VMWare Workstation provider](https://www.vagrantup.com/vmware) - 4. [Parallels Desktop](https://www.parallels.com/products/desktop/) version 9 or greater as well as the [Vagrant Parallels provider](https://parallels.github.io/vagrant-parallels/) - 5. libvirt with KVM and enable support of hardware virtualisation. [Vagrant-libvirt](https://github.com/pradels/vagrant-libvirt). For fedora provided official rpm, and possible to use `yum install vagrant-libvirt` - -### Setup - -Setting up a cluster is as simple as running: - -{% highlight sh %} -{% raw %} -export KUBERNETES_PROVIDER=vagrant -curl -sS https://get.k8s.io | bash -{% endraw %} -{% endhighlight %} - -Alternatively, you can download [Kubernetes release](https://github.com/kubernetes/kubernetes/releases) and extract the archive. To start your local cluster, open a shell and run: - -{% highlight sh %} -{% raw %} -cd kubernetes - -export KUBERNETES_PROVIDER=vagrant -./cluster/kube-up.sh -{% endraw %} -{% endhighlight %} - -The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster management scripts which variant to use. If you forget to set this, the assumption is you are running on Google Compute Engine. - -By default, the Vagrant setup will create a single master VM (called kubernetes-master) and one node (called kubernetes-minion-1). Each VM will take 1 GB, so make sure you have at least 2GB to 4GB of free memory (plus appropriate free disk space). - -Vagrant will provision each machine in the cluster with all the necessary components to run Kubernetes. The initial setup can take a few minutes to complete on each machine. - -If you installed more than one Vagrant provider, Kubernetes will usually pick the appropriate one. However, you can override which one Kubernetes will use by setting the [`VAGRANT_DEFAULT_PROVIDER`](https://docs.vagrantup.com/v2/providers/default.html) environment variable: - -{% highlight sh %} -{% raw %} -export VAGRANT_DEFAULT_PROVIDER=parallels -export KUBERNETES_PROVIDER=vagrant -./cluster/kube-up.sh -{% endraw %} -{% endhighlight %} - -By default, each VM in the cluster is running Fedora. - -To access the master or any node: - -{% highlight sh %} -{% raw %} -vagrant ssh master -vagrant ssh minion-1 -{% endraw %} -{% endhighlight %} - -If you are running more than one node, you can access the others by: - -{% highlight sh %} -{% raw %} -vagrant ssh minion-2 -vagrant ssh minion-3 -{% endraw %} -{% endhighlight %} - -Each node in the cluster installs the docker daemon and the kubelet. - -The master node instantiates the Kubernetes master components as pods on the machine. - -To view the service status and/or logs on the kubernetes-master: - -{% highlight console %} -{% raw %} -[vagrant@kubernetes-master ~] $ vagrant ssh master -[vagrant@kubernetes-master ~] $ sudo su - -[root@kubernetes-master ~] $ systemctl status kubelet -[root@kubernetes-master ~] $ journalctl -ru kubelet - -[root@kubernetes-master ~] $ systemctl status docker -[root@kubernetes-master ~] $ journalctl -ru docker - -[root@kubernetes-master ~] $ tail -f /var/log/kube-apiserver.log -[root@kubernetes-master ~] $ tail -f /var/log/kube-controller-manager.log -[root@kubernetes-master ~] $ tail -f /var/log/kube-scheduler.log -{% endraw %} -{% endhighlight %} - -To view the services on any of the nodes: - -{% highlight console %} -{% raw %} -[vagrant@kubernetes-master ~] $ vagrant ssh minion-1 -[vagrant@kubernetes-master ~] $ sudo su - -[root@kubernetes-master ~] $ systemctl status kubelet -[root@kubernetes-master ~] $ journalctl -ru kubelet - -[root@kubernetes-master ~] $ systemctl status docker -[root@kubernetes-master ~] $ journalctl -ru docker -{% endraw %} -{% endhighlight %} - -### Interacting with your Kubernetes cluster with Vagrant. - -With your Kubernetes cluster up, you can manage the nodes in your cluster with the regular Vagrant commands. - -To push updates to new Kubernetes code after making source changes: - -{% highlight sh %} -{% raw %} -./cluster/kube-push.sh -{% endraw %} -{% endhighlight %} - -To stop and then restart the cluster: - -{% highlight sh %} -{% raw %} -vagrant halt -./cluster/kube-up.sh -{% endraw %} -{% endhighlight %} - -To destroy the cluster: - -{% highlight sh %} -{% raw %} -vagrant destroy -{% endraw %} -{% endhighlight %} - -Once your Vagrant machines are up and provisioned, the first thing to do is to check that you can use the `kubectl.sh` script. - -You may need to build the binaries first, you can do this with `make` - -{% highlight console %} -{% raw %} -$ ./cluster/kubectl.sh get nodes - -NAME LABELS -10.245.1.4 -10.245.1.5 -10.245.1.3 -{% endraw %} -{% endhighlight %} - -### Authenticating with your master - -When using the vagrant provider in Kubernetes, the `cluster/kubectl.sh` script will cache your credentials in a `~/.kubernetes_vagrant_auth` file so you will not be prompted for them in the future. - -{% highlight sh %} -{% raw %} -cat ~/.kubernetes_vagrant_auth -{% endraw %} -{% endhighlight %} - -{% highlight json %} -{% raw %} -{ "User": "vagrant", - "Password": "vagrant", - "CAFile": "/home/k8s_user/.kubernetes.vagrant.ca.crt", - "CertFile": "/home/k8s_user/.kubecfg.vagrant.crt", - "KeyFile": "/home/k8s_user/.kubecfg.vagrant.key" -} -{% endraw %} -{% endhighlight %} - -You should now be set to use the `cluster/kubectl.sh` script. For example try to list the nodes that you have started with: - -{% highlight sh %} -{% raw %} -./cluster/kubectl.sh get nodes -{% endraw %} -{% endhighlight %} - -### Running containers - -Your cluster is running, you can list the nodes in your cluster: - -{% highlight console %} -{% raw %} -$ ./cluster/kubectl.sh get nodes - -NAME LABELS -10.245.2.4 -10.245.2.3 -10.245.2.2 -{% endraw %} -{% endhighlight %} - -Now start running some containers! - -You can now use any of the `cluster/kube-*.sh` commands to interact with your VM machines. -Before starting a container there will be no pods, services and replication controllers. - -{% highlight console %} -{% raw %} -$ ./cluster/kubectl.sh get pods -NAME READY STATUS RESTARTS AGE - -$ ./cluster/kubectl.sh get services -NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE - -$ ./cluster/kubectl.sh get replicationcontrollers -CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -{% endraw %} -{% endhighlight %} - -Start a container running nginx with a replication controller and three replicas - -{% highlight console %} -{% raw %} -$ ./cluster/kubectl.sh run my-nginx --image=nginx --replicas=3 --port=80 -{% endraw %} -{% endhighlight %} - -When listing the pods, you will see that three containers have been started and are in Waiting state: - -{% highlight console %} -{% raw %} -$ ./cluster/kubectl.sh get pods -NAME READY STATUS RESTARTS AGE -my-nginx-5kq0g 0/1 Pending 0 10s -my-nginx-gr3hh 0/1 Pending 0 10s -my-nginx-xql4j 0/1 Pending 0 10s -{% endraw %} -{% endhighlight %} - -You need to wait for the provisioning to complete, you can monitor the nodes by doing: - -{% highlight console %} -{% raw %} -$ vagrant ssh minion-1 -c 'sudo docker images' -kubernetes-minion-1: - REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE - 96864a7d2df3 26 hours ago 204.4 MB - google/cadvisor latest e0575e677c50 13 days ago 12.64 MB - kubernetes/pause latest 6c4579af347b 8 weeks ago 239.8 kB -{% endraw %} -{% endhighlight %} - -Once the docker image for nginx has been downloaded, the container will start and you can list it: - -{% highlight console %} -{% raw %} -$ vagrant ssh minion-1 -c 'sudo docker ps' -kubernetes-minion-1: - CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - dbe79bf6e25b nginx:latest "nginx" 21 seconds ago Up 19 seconds k8s--mynginx.8c5b8a3a--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--fcfa837f - fa0e29c94501 kubernetes/pause:latest "/pause" 8 minutes ago Up 8 minutes 0.0.0.0:8080->80/tcp k8s--net.a90e7ce4--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--baf5b21b - aa2ee3ed844a google/cadvisor:latest "/usr/bin/cadvisor" 38 minutes ago Up 38 minutes k8s--cadvisor.9e90d182--cadvisor_-_agent.file--4626b3a2 - 65a3a926f357 kubernetes/pause:latest "/pause" 39 minutes ago Up 39 minutes 0.0.0.0:4194->8080/tcp k8s--net.c5ba7f0e--cadvisor_-_agent.file--342fd561 -{% endraw %} -{% endhighlight %} - -Going back to listing the pods, services and replicationcontrollers, you now have: - -{% highlight console %} -{% raw %} -$ ./cluster/kubectl.sh get pods -NAME READY STATUS RESTARTS AGE -my-nginx-5kq0g 1/1 Running 0 1m -my-nginx-gr3hh 1/1 Running 0 1m -my-nginx-xql4j 1/1 Running 0 1m - -$ ./cluster/kubectl.sh get services -NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE -my-nginx 10.0.0.1 80/TCP run=my-nginx 1h -{% endraw %} -{% endhighlight %} - -We did not start any services, hence there are none listed. But we see three replicas displayed properly. -Check the [guestbook](../../examples/guestbook/README.html) application to learn how to create a service. -You can already play with scaling the replicas with: - -{% highlight console %} -{% raw %} -$ ./cluster/kubectl.sh scale rc my-nginx --replicas=2 -$ ./cluster/kubectl.sh get pods -NAME READY STATUS RESTARTS AGE -my-nginx-5kq0g 1/1 Running 0 2m -my-nginx-gr3hh 1/1 Running 0 2m -{% endraw %} -{% endhighlight %} - -Congratulations! - -### Troubleshooting - -#### I keep downloading the same (large) box all the time! - -By default the Vagrantfile will download the box from S3. You can change this (and cache the box locally) by providing a name and an alternate URL when calling `kube-up.sh` - -{% highlight sh %} -{% raw %} -export KUBERNETES_BOX_NAME=choose_your_own_name_for_your_kuber_box -export KUBERNETES_BOX_URL=path_of_your_kuber_box -export KUBERNETES_PROVIDER=vagrant -./cluster/kube-up.sh -{% endraw %} -{% endhighlight %} - -#### I just created the cluster, but I am getting authorization errors! - -You probably have an incorrect ~/.kubernetes_vagrant_auth file for the cluster you are attempting to contact. - -{% highlight sh %} -{% raw %} -rm ~/.kubernetes_vagrant_auth -{% endraw %} -{% endhighlight %} - -After using kubectl.sh make sure that the correct credentials are set: - -{% highlight sh %} -{% raw %} -cat ~/.kubernetes_vagrant_auth -{% endraw %} -{% endhighlight %} - -{% highlight json %} -{% raw %} -{ - "User": "vagrant", - "Password": "vagrant" -} -{% endraw %} -{% endhighlight %} - -#### I just created the cluster, but I do not see my container running! - -If this is your first time creating the cluster, the kubelet on each node schedules a number of docker pull requests to fetch prerequisite images. This can take some time and as a result may delay your initial pod getting provisioned. - -#### I want to make changes to Kubernetes code! - -To set up a vagrant cluster for hacking, follow the [vagrant developer guide](../devel/developer-guides/vagrant.html). - -#### I have brought Vagrant up but the nodes cannot validate! - -Log on to one of the nodes (`vagrant ssh minion-1`) and inspect the salt minion log (`sudo cat /var/log/salt/minion`). - -#### I want to change the number of nodes! - -You can control the number of nodes that are instantiated via the environment variable `NUM_MINIONS` on your host machine. If you plan to work with replicas, we strongly encourage you to work with enough nodes to satisfy your largest intended replica size. If you do not plan to work with replicas, you can save some system resources by running with a single node. You do this, by setting `NUM_MINIONS` to 1 like so: - -{% highlight sh %} -{% raw %} -export NUM_MINIONS=1 -{% endraw %} -{% endhighlight %} - -#### I want my VMs to have more memory! - -You can control the memory allotted to virtual machines with the `KUBERNETES_MEMORY` environment variable. -Just set it to the number of megabytes you would like the machines to have. For example: - -{% highlight sh %} -{% raw %} -export KUBERNETES_MEMORY=2048 -{% endraw %} -{% endhighlight %} - -If you need more granular control, you can set the amount of memory for the master and nodes independently. For example: - -{% highlight sh %} -{% raw %} -export KUBERNETES_MASTER_MEMORY=1536 -export KUBERNETES_MINION_MEMORY=2048 -{% endraw %} -{% endhighlight %} - -#### I ran vagrant suspend and nothing works! - -`vagrant suspend` seems to mess up the network. This is not supported at this time. - -#### I want vagrant to sync folders via nfs! - -You can ensure that vagrant uses nfs to sync folders with virtual machines by setting the KUBERNETES_VAGRANT_USE_NFS environment variable to 'true'. nfs is faster than virtualbox or vmware's 'shared folders' and does not require guest additions. See the [vagrant docs](http://docs.vagrantup.com/v2/synced-folders/nfs.html) for details on configuring nfs on the host. This setting will have no effect on the libvirt provider, which uses nfs by default. For example: - -{% highlight sh %} -{% raw %} -export KUBERNETES_VAGRANT_USE_NFS=true -{% endraw %} +--- +title: "Getting started with Vagrant" +--- + +Running Kubernetes with Vagrant (and VirtualBox) is an easy way to run/test/develop on your local machine (Linux, Mac OS X). + + + +{% include pagetoc.html %} + +### Prerequisites + +1. Install latest version >= 1.6.2 of vagrant from http://www.vagrantup.com/downloads.html +2. Install one of: + 1. Version 4.3.28 of Virtual Box from https://www.virtualbox.org/wiki/Download_Old_Builds_4_3 + 2. [VMWare Fusion](https://www.vmware.com/products/fusion/) version 5 or greater as well as the appropriate [Vagrant VMWare Fusion provider](https://www.vagrantup.com/vmware) + 3. [VMWare Workstation](https://www.vmware.com/products/workstation/) version 9 or greater as well as the [Vagrant VMWare Workstation provider](https://www.vagrantup.com/vmware) + 4. [Parallels Desktop](https://www.parallels.com/products/desktop/) version 9 or greater as well as the [Vagrant Parallels provider](https://parallels.github.io/vagrant-parallels/) + 5. libvirt with KVM and enable support of hardware virtualisation. [Vagrant-libvirt](https://github.com/pradels/vagrant-libvirt). For fedora provided official rpm, and possible to use `yum install vagrant-libvirt` + +### Setup + +Setting up a cluster is as simple as running: + +{% highlight sh %} +export KUBERNETES_PROVIDER=vagrant +curl -sS https://get.k8s.io | bash +{% endhighlight %} + +Alternatively, you can download [Kubernetes release](https://github.com/kubernetes/kubernetes/releases) and extract the archive. To start your local cluster, open a shell and run: + +{% highlight sh %} +cd kubernetes + +export KUBERNETES_PROVIDER=vagrant +./cluster/kube-up.sh +{% endhighlight %} + +The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster management scripts which variant to use. If you forget to set this, the assumption is you are running on Google Compute Engine. + +By default, the Vagrant setup will create a single master VM (called kubernetes-master) and one node (called kubernetes-minion-1). Each VM will take 1 GB, so make sure you have at least 2GB to 4GB of free memory (plus appropriate free disk space). + +Vagrant will provision each machine in the cluster with all the necessary components to run Kubernetes. The initial setup can take a few minutes to complete on each machine. + +If you installed more than one Vagrant provider, Kubernetes will usually pick the appropriate one. However, you can override which one Kubernetes will use by setting the [`VAGRANT_DEFAULT_PROVIDER`](https://docs.vagrantup.com/v2/providers/default) environment variable: + +{% highlight sh %} +export VAGRANT_DEFAULT_PROVIDER=parallels +export KUBERNETES_PROVIDER=vagrant +./cluster/kube-up.sh +{% endhighlight %} + +By default, each VM in the cluster is running Fedora. + +To access the master or any node: + +{% highlight sh %} +vagrant ssh master +vagrant ssh minion-1 +{% endhighlight %} + +If you are running more than one node, you can access the others by: + +{% highlight sh %} +vagrant ssh minion-2 +vagrant ssh minion-3 +{% endhighlight %} + +Each node in the cluster installs the docker daemon and the kubelet. + +The master node instantiates the Kubernetes master components as pods on the machine. + +To view the service status and/or logs on the kubernetes-master: + +{% highlight console %} +[vagrant@kubernetes-master ~] $ vagrant ssh master +[vagrant@kubernetes-master ~] $ sudo su + +[root@kubernetes-master ~] $ systemctl status kubelet +[root@kubernetes-master ~] $ journalctl -ru kubelet + +[root@kubernetes-master ~] $ systemctl status docker +[root@kubernetes-master ~] $ journalctl -ru docker + +[root@kubernetes-master ~] $ tail -f /var/log/kube-apiserver.log +[root@kubernetes-master ~] $ tail -f /var/log/kube-controller-manager.log +[root@kubernetes-master ~] $ tail -f /var/log/kube-scheduler.log +{% endhighlight %} + +To view the services on any of the nodes: + +{% highlight console %} +[vagrant@kubernetes-master ~] $ vagrant ssh minion-1 +[vagrant@kubernetes-master ~] $ sudo su + +[root@kubernetes-master ~] $ systemctl status kubelet +[root@kubernetes-master ~] $ journalctl -ru kubelet + +[root@kubernetes-master ~] $ systemctl status docker +[root@kubernetes-master ~] $ journalctl -ru docker +{% endhighlight %} + +### Interacting with your Kubernetes cluster with Vagrant. + +With your Kubernetes cluster up, you can manage the nodes in your cluster with the regular Vagrant commands. + +To push updates to new Kubernetes code after making source changes: + +{% highlight sh %} +./cluster/kube-push.sh +{% endhighlight %} + +To stop and then restart the cluster: + +{% highlight sh %} +vagrant halt +./cluster/kube-up.sh +{% endhighlight %} + +To destroy the cluster: + +{% highlight sh %} +vagrant destroy +{% endhighlight %} + +Once your Vagrant machines are up and provisioned, the first thing to do is to check that you can use the `kubectl.sh` script. + +You may need to build the binaries first, you can do this with `make` + +{% highlight console %} +$ ./cluster/kubectl.sh get nodes + +NAME LABELS +10.245.1.4 +10.245.1.5 +10.245.1.3 +{% endhighlight %} + +### Authenticating with your master + +When using the vagrant provider in Kubernetes, the `cluster/kubectl.sh` script will cache your credentials in a `~/.kubernetes_vagrant_auth` file so you will not be prompted for them in the future. + +{% highlight sh %} +cat ~/.kubernetes_vagrant_auth +{% endhighlight %} + +{% highlight json %} +{ "User": "vagrant", + "Password": "vagrant", + "CAFile": "/home/k8s_user/.kubernetes.vagrant.ca.crt", + "CertFile": "/home/k8s_user/.kubecfg.vagrant.crt", + "KeyFile": "/home/k8s_user/.kubecfg.vagrant.key" +} +{% endhighlight %} + +You should now be set to use the `cluster/kubectl.sh` script. For example try to list the nodes that you have started with: + +{% highlight sh %} +./cluster/kubectl.sh get nodes +{% endhighlight %} + +### Running containers + +Your cluster is running, you can list the nodes in your cluster: + +{% highlight console %} +$ ./cluster/kubectl.sh get nodes + +NAME LABELS +10.245.2.4 +10.245.2.3 +10.245.2.2 +{% endhighlight %} + +Now start running some containers! + +You can now use any of the `cluster/kube-*.sh` commands to interact with your VM machines. +Before starting a container there will be no pods, services and replication controllers. + +{% highlight console %} +$ ./cluster/kubectl.sh get pods +NAME READY STATUS RESTARTS AGE + +$ ./cluster/kubectl.sh get services +NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE + +$ ./cluster/kubectl.sh get replicationcontrollers +CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS +{% endhighlight %} + +Start a container running nginx with a replication controller and three replicas + +{% highlight console %} +$ ./cluster/kubectl.sh run my-nginx --image=nginx --replicas=3 --port=80 +{% endhighlight %} + +When listing the pods, you will see that three containers have been started and are in Waiting state: + +{% highlight console %} +$ ./cluster/kubectl.sh get pods +NAME READY STATUS RESTARTS AGE +my-nginx-5kq0g 0/1 Pending 0 10s +my-nginx-gr3hh 0/1 Pending 0 10s +my-nginx-xql4j 0/1 Pending 0 10s +{% endhighlight %} + +You need to wait for the provisioning to complete, you can monitor the nodes by doing: + +{% highlight console %} +$ vagrant ssh minion-1 -c 'sudo docker images' +kubernetes-minion-1: + REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE + 96864a7d2df3 26 hours ago 204.4 MB + google/cadvisor latest e0575e677c50 13 days ago 12.64 MB + kubernetes/pause latest 6c4579af347b 8 weeks ago 239.8 kB +{% endhighlight %} + +Once the docker image for nginx has been downloaded, the container will start and you can list it: + +{% highlight console %} +$ vagrant ssh minion-1 -c 'sudo docker ps' +kubernetes-minion-1: + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + dbe79bf6e25b nginx:latest "nginx" 21 seconds ago Up 19 seconds k8s--mynginx.8c5b8a3a--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--fcfa837f + fa0e29c94501 kubernetes/pause:latest "/pause" 8 minutes ago Up 8 minutes 0.0.0.0:8080->80/tcp k8s--net.a90e7ce4--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--baf5b21b + aa2ee3ed844a google/cadvisor:latest "/usr/bin/cadvisor" 38 minutes ago Up 38 minutes k8s--cadvisor.9e90d182--cadvisor_-_agent.file--4626b3a2 + 65a3a926f357 kubernetes/pause:latest "/pause" 39 minutes ago Up 39 minutes 0.0.0.0:4194->8080/tcp k8s--net.c5ba7f0e--cadvisor_-_agent.file--342fd561 +{% endhighlight %} + +Going back to listing the pods, services and replicationcontrollers, you now have: + +{% highlight console %} +$ ./cluster/kubectl.sh get pods +NAME READY STATUS RESTARTS AGE +my-nginx-5kq0g 1/1 Running 0 1m +my-nginx-gr3hh 1/1 Running 0 1m +my-nginx-xql4j 1/1 Running 0 1m + +$ ./cluster/kubectl.sh get services +NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE +my-nginx 10.0.0.1 80/TCP run=my-nginx 1h +{% endhighlight %} + +We did not start any services, hence there are none listed. But we see three replicas displayed properly. +Check the [guestbook](../../examples/guestbook/README) application to learn how to create a service. +You can already play with scaling the replicas with: + +{% highlight console %} +$ ./cluster/kubectl.sh scale rc my-nginx --replicas=2 +$ ./cluster/kubectl.sh get pods +NAME READY STATUS RESTARTS AGE +my-nginx-5kq0g 1/1 Running 0 2m +my-nginx-gr3hh 1/1 Running 0 2m +{% endhighlight %} + +Congratulations! + +### Troubleshooting + +#### I keep downloading the same (large) box all the time! + +By default the Vagrantfile will download the box from S3. You can change this (and cache the box locally) by providing a name and an alternate URL when calling `kube-up.sh` + +{% highlight sh %} +export KUBERNETES_BOX_NAME=choose_your_own_name_for_your_kuber_box +export KUBERNETES_BOX_URL=path_of_your_kuber_box +export KUBERNETES_PROVIDER=vagrant +./cluster/kube-up.sh +{% endhighlight %} + +#### I just created the cluster, but I am getting authorization errors! + +You probably have an incorrect ~/.kubernetes_vagrant_auth file for the cluster you are attempting to contact. + +{% highlight sh %} +rm ~/.kubernetes_vagrant_auth +{% endhighlight %} + +After using kubectl.sh make sure that the correct credentials are set: + +{% highlight sh %} +cat ~/.kubernetes_vagrant_auth +{% endhighlight %} + +{% highlight json %} +{ + "User": "vagrant", + "Password": "vagrant" +} +{% endhighlight %} + +#### I just created the cluster, but I do not see my container running! + +If this is your first time creating the cluster, the kubelet on each node schedules a number of docker pull requests to fetch prerequisite images. This can take some time and as a result may delay your initial pod getting provisioned. + +#### I want to make changes to Kubernetes code! + +To set up a vagrant cluster for hacking, follow the [vagrant developer guide](../devel/developer-guides/vagrant). + +#### I have brought Vagrant up but the nodes cannot validate! + +Log on to one of the nodes (`vagrant ssh minion-1`) and inspect the salt minion log (`sudo cat /var/log/salt/minion`). + +#### I want to change the number of nodes! + +You can control the number of nodes that are instantiated via the environment variable `NUM_MINIONS` on your host machine. If you plan to work with replicas, we strongly encourage you to work with enough nodes to satisfy your largest intended replica size. If you do not plan to work with replicas, you can save some system resources by running with a single node. You do this, by setting `NUM_MINIONS` to 1 like so: + +{% highlight sh %} +export NUM_MINIONS=1 +{% endhighlight %} + +#### I want my VMs to have more memory! + +You can control the memory allotted to virtual machines with the `KUBERNETES_MEMORY` environment variable. +Just set it to the number of megabytes you would like the machines to have. For example: + +{% highlight sh %} +export KUBERNETES_MEMORY=2048 +{% endhighlight %} + +If you need more granular control, you can set the amount of memory for the master and nodes independently. For example: + +{% highlight sh %} +export KUBERNETES_MASTER_MEMORY=1536 +export KUBERNETES_MINION_MEMORY=2048 +{% endhighlight %} + +#### I ran vagrant suspend and nothing works! + +`vagrant suspend` seems to mess up the network. This is not supported at this time. + +#### I want vagrant to sync folders via nfs! + +You can ensure that vagrant uses nfs to sync folders with virtual machines by setting the KUBERNETES_VAGRANT_USE_NFS environment variable to 'true'. nfs is faster than virtualbox or vmware's 'shared folders' and does not require guest additions. See the [vagrant docs](http://docs.vagrantup.com/v2/synced-folders/nfs) for details on configuring nfs on the host. This setting will have no effect on the libvirt provider, which uses nfs by default. For example: + +{% highlight sh %} +export KUBERNETES_VAGRANT_USE_NFS=true {% endhighlight %} \ No newline at end of file diff --git a/v1.1/docs/getting-started-guides/vsphere.md b/v1.1/docs/getting-started-guides/vsphere.md index 86673adbac..199b84ca79 100644 --- a/v1.1/docs/getting-started-guides/vsphere.md +++ b/v1.1/docs/getting-started-guides/vsphere.md @@ -1,98 +1,85 @@ ---- -title: "Getting started with vSphere" -section: guides ---- -The example below creates a Kubernetes cluster with 4 worker node Virtual -Machines and a master Virtual Machine (i.e. 5 VMs in your cluster). This -cluster is set up and controlled from your workstation (or wherever you find -convenient). - -## Table of Contents - -{% include pagetoc.html %} - -### Prerequisites - -1. You need administrator credentials to an ESXi machine or vCenter instance. -2. You must have Go (version 1.2 or later) installed: [www.golang.org](http://www.golang.org). -3. You must have your `GOPATH` set up and include `$GOPATH/bin` in your `PATH`. - -{% highlight sh %} -{% raw %} - export GOPATH=$HOME/src/go - mkdir -p $GOPATH - export PATH=$PATH:$GOPATH/bin -{% endraw %} -{% endhighlight %} - -4. Install the govc tool to interact with ESXi/vCenter: - -{% highlight sh %} -{% raw %} - go get github.com/vmware/govmomi/govc -{% endraw %} -{% endhighlight %} - -5. Get or build a [binary release](binary_release.html) - -### Setup - -Download a prebuilt Debian 7.7 VMDK that we'll use as a base image: - -{% highlight sh %} -{% raw %} -curl --remote-name-all https://storage.googleapis.com/govmomi/vmdk/2014-11-11/kube.vmdk.gz{,.md5} -md5sum -c kube.vmdk.gz.md5 -gzip -d kube.vmdk.gz -{% endraw %} -{% endhighlight %} - -Import this VMDK into your vSphere datastore: - -{% highlight sh %} -{% raw %} -export GOVC_URL='user:pass@hostname' -export GOVC_INSECURE=1 # If the host above uses a self-signed cert -export GOVC_DATASTORE='target datastore' -export GOVC_RESOURCE_POOL='resource pool or cluster with access to datastore' - -govc import.vmdk kube.vmdk ./kube/ -{% endraw %} -{% endhighlight %} - -Verify that the VMDK was correctly uploaded and expanded to ~3GiB: - -{% highlight sh %} -{% raw %} -govc datastore.ls ./kube/ -{% endraw %} -{% endhighlight %} - -Take a look at the file `cluster/vsphere/config-common.sh` fill in the required -parameters. The guest login for the image that you imported is `kube:kube`. - -### Starting a cluster - -Now, let's continue with deploying Kubernetes. -This process takes about ~10 minutes. - -{% highlight sh %} -{% raw %} -cd kubernetes # Extracted binary release OR repository root -export KUBERNETES_PROVIDER=vsphere -cluster/kube-up.sh -{% endraw %} -{% endhighlight %} - -Refer to the top level README and the getting started guide for Google Compute -Engine. Once you have successfully reached this point, your vSphere Kubernetes -deployment works just as any other one! - -**Enjoy!** - -### Extra: debugging deployment failure - -The output of `kube-up.sh` displays the IP addresses of the VMs it deploys. You -can log into any VM as the `kube` user to poke around and figure out what is -going on (find yourself authorized with your SSH key, or use the password +--- +title: "Getting started with vSphere" +--- +The example below creates a Kubernetes cluster with 4 worker node Virtual +Machines and a master Virtual Machine (i.e. 5 VMs in your cluster). This +cluster is set up and controlled from your workstation (or wherever you find +convenient). + + + +{% include pagetoc.html %} + +### Prerequisites + +1. You need administrator credentials to an ESXi machine or vCenter instance. +2. You must have Go (version 1.2 or later) installed: [www.golang.org](http://www.golang.org). +3. You must have your `GOPATH` set up and include `$GOPATH/bin` in your `PATH`. + +{% highlight sh %} + export GOPATH=$HOME/src/go + mkdir -p $GOPATH + export PATH=$PATH:$GOPATH/bin +{% endhighlight %} + +4. Install the govc tool to interact with ESXi/vCenter: + +{% highlight sh %} + go get github.com/vmware/govmomi/govc +{% endhighlight %} + +5. Get or build a [binary release](binary_release) + +### Setup + +Download a prebuilt Debian 7.7 VMDK that we'll use as a base image: + +{% highlight sh %} +curl --remote-name-all https://storage.googleapis.com/govmomi/vmdk/2014-11-11/kube.vmdk.gz{,.md5} +md5sum -c kube.vmdk.gz.md5 +gzip -d kube.vmdk.gz +{% endhighlight %} + +Import this VMDK into your vSphere datastore: + +{% highlight sh %} +export GOVC_URL='user:pass@hostname' +export GOVC_INSECURE=1 # If the host above uses a self-signed cert +export GOVC_DATASTORE='target datastore' +export GOVC_RESOURCE_POOL='resource pool or cluster with access to datastore' + +govc import.vmdk kube.vmdk ./kube/ +{% endhighlight %} + +Verify that the VMDK was correctly uploaded and expanded to ~3GiB: + +{% highlight sh %} +govc datastore.ls ./kube/ +{% endhighlight %} + +Take a look at the file `cluster/vsphere/config-common.sh` fill in the required +parameters. The guest login for the image that you imported is `kube:kube`. + +### Starting a cluster + +Now, let's continue with deploying Kubernetes. +This process takes about ~10 minutes. + +{% highlight sh %} +cd kubernetes # Extracted binary release OR repository root +export KUBERNETES_PROVIDER=vsphere +cluster/kube-up.sh +{% endhighlight %} + +Refer to the top level README and the getting started guide for Google Compute +Engine. Once you have successfully reached this point, your vSphere Kubernetes +deployment works just as any other one! + +**Enjoy!** + +### Extra: debugging deployment failure + +The output of `kube-up.sh` displays the IP addresses of the VMs it deploys. You +can log into any VM as the `kube` user to poke around and figure out what is +going on (find yourself authorized with your SSH key, or use the password `kube` otherwise). \ No newline at end of file diff --git a/v1.1/docs/index.md b/v1.1/docs/index.md index 0f06e68e06..abed6e0046 100644 --- a/v1.1/docs/index.md +++ b/v1.1/docs/index.md @@ -1,21 +1,17 @@ --- title: "Kubernetes Documentation: releases.k8s.io/release-1.1" --- - - -# Kubernetes Documentation: releases.k8s.io/release-1.1 - -* The [User's guide](user-guide/README.html) is for anyone who wants to run programs and +* The [User's guide](user-guide/README) is for anyone who wants to run programs and services on an existing Kubernetes cluster. -* The [Cluster Admin's guide](admin/README.html) is for anyone setting up +* The [Cluster Admin's guide](admin/README) is for anyone setting up a Kubernetes cluster or administering it. -* The [Developer guide](devel/README.html) is for anyone wanting to write +* The [Developer guide](devel/README) is for anyone wanting to write programs that access the Kubernetes API, write plugins or extensions, or modify the core code of Kubernetes. -* The [Kubectl Command Line Interface](user-guide/kubectl/kubectl.html) is a detailed reference on +* The [Kubectl Command Line Interface](user-guide/kubectl/kubectl) is a detailed reference on the `kubectl` CLI. * The [API object documentation](http://kubernetes.io/third_party/swagger-ui/) @@ -26,10 +22,10 @@ title: "Kubernetes Documentation: releases.k8s.io/release-1.1" * There are example files and walkthroughs in the [examples](../examples/) folder. -* If something went wrong, see the [troubleshooting](troubleshooting.html) document for how to debug. -You should also check the [known issues](user-guide/known-issues.html) for the release you're using. +* If something went wrong, see the [troubleshooting](troubleshooting) document for how to debug. +You should also check the [known issues](user-guide/known-issues) for the release you're using. -* To report a security issue, see [Reporting a Security Issue](reporting-security-issues.html). +* To report a security issue, see [Reporting a Security Issue](reporting-security-issues). diff --git a/v1.1/docs/reporting-security-issues.md b/v1.1/docs/reporting-security-issues.md index 8d9253593e..5cdd692303 100644 --- a/v1.1/docs/reporting-security-issues.md +++ b/v1.1/docs/reporting-security-issues.md @@ -1,10 +1,6 @@ --- title: "Security" --- - - -# Security - If you believe you have discovered a vulnerability or a have a security incident to report, please follow the steps below. This applies to Kubernetes releases v1.0 or later. To watch for security and major API announcements, please join our [kubernetes-announce](https://groups.google.com/forum/#!forum/kubernetes-announce) group. @@ -13,8 +9,8 @@ To watch for security and major API announcements, please join our [kubernetes-a To report an issue, please: - Submit a bug report [here](http://goo.gl/vulnz). - - Select “I want to report a technical security bug in a Google product (SQLi, XSS, etc.).â€? - - Select “Otherâ€? as the Application Type. + - Select 'I want to report a technical security bug in a Google product (SQLi, XSS, etc.).'? + - Select 'Other'? as the Application Type. - Under reproduction steps, please additionally include - the words "Kubernetes Security issue" - Description of the issue diff --git a/v1.1/docs/roadmap.md b/v1.1/docs/roadmap.md index 12f1895638..044ec3076a 100644 --- a/v1.1/docs/roadmap.md +++ b/v1.1/docs/roadmap.md @@ -1,10 +1,6 @@ --- title: "Kubernetes Roadmap" --- - - -# Kubernetes Roadmap - ## Kubernetes 1.1 ### Timeline diff --git a/v1.1/docs/troubleshooting.md b/v1.1/docs/troubleshooting.md index fd95e9fa2e..8dcf672247 100644 --- a/v1.1/docs/troubleshooting.md +++ b/v1.1/docs/troubleshooting.md @@ -1,15 +1,11 @@ --- title: "Troubleshooting" --- - - -# Troubleshooting - Sometimes things go wrong. This guide is aimed at making them right. It has two sections: - * [Troubleshooting your application](user-guide/application-troubleshooting.html) - Useful for users who are deploying code into Kubernetes and wondering why it is not working. - * [Troubleshooting your cluster](admin/cluster-troubleshooting.html) - Useful for cluster administrators and people whose Kubernetes cluster is unhappy. + * [Troubleshooting your application](user-guide/application-troubleshooting) - Useful for users who are deploying code into Kubernetes and wondering why it is not working. + * [Troubleshooting your cluster](admin/cluster-troubleshooting) - Useful for cluster administrators and people whose Kubernetes cluster is unhappy. -You should also check the [known issues](user-guide/known-issues.html) for the release you're using. +You should also check the [known issues](user-guide/known-issues) for the release you're using. # Getting help @@ -17,7 +13,7 @@ If your problem isn't answered by any of the guides above, there are variety of ## Questions -If you aren't familiar with it, many of your questions may be answered by the [user guide](user-guide/README.html). +If you aren't familiar with it, many of your questions may be answered by the [user guide](user-guide/README). We also have a number of FAQ pages: * [User FAQ](https://github.com/kubernetes/kubernetes/wiki/User-FAQ) diff --git a/v1.1/docs/user-guide/README.md b/v1.1/docs/user-guide/README.md index 5aa731f8ee..d253184575 100644 --- a/v1.1/docs/user-guide/README.md +++ b/v1.1/docs/user-guide/README.md @@ -1,103 +1,90 @@ --- title: "Kubernetes User Guide: Managing Applications" --- +{% include pagetoc.html %} +The user guide is intended for anyone who wants to run programs and services on an existing Kubernetes cluster. Setup and administration of a Kubernetes cluster is described in the [Cluster Admin Guide](/{{page.version}}/docs/admin/README). The [Developer Guide](/{{page.version}}/docs/devel/README) is for anyone wanting to either write code which directly accesses the Kubernetes API, or to contribute directly to the Kubernetes project. -# Kubernetes User Guide: Managing Applications - -**Table of Contents** - - -- [Kubernetes User Guide: Managing Applications](#kubernetes-user-guide-managing-applications) - - [Quick walkthrough](#quick-walkthrough) - - [Thorough walkthrough](#thorough-walkthrough) - - [Concept guide](#concept-guide) - - [Further reading](#further-reading) - - - -The user guide is intended for anyone who wants to run programs and services on an existing Kubernetes cluster. Setup and administration of a Kubernetes cluster is described in the [Cluster Admin Guide](../../docs/admin/README.html). The [Developer Guide](../../docs/devel/README.html) is for anyone wanting to either write code which directly accesses the Kubernetes API, or to contribute directly to the Kubernetes project. - -Please ensure you have completed the [prerequisites for running examples from the user guide](prereqs.html). +Please ensure you have completed the [prerequisites for running examples from the user guide](prereqs). ## Quick walkthrough -1. [Kubernetes 101](walkthrough/README.html) -1. [Kubernetes 201](walkthrough/k8s201.html) +1. [Kubernetes 101](walkthrough/README) +1. [Kubernetes 201](walkthrough/k8s201) ## Thorough walkthrough If you don't have much familiarity with Kubernetes, we recommend you read the following sections in order: -1. [Quick start: launch and expose an application](quick-start.html) -1. [Configuring and launching containers: configuring common container parameters](configuring-containers.html) -1. [Deploying continuously running applications](deploying-applications.html) -1. [Connecting applications: exposing applications to clients and users](connecting-applications.html) -1. [Working with containers in production](production-pods.html) -1. [Managing deployments](managing-deployments.html) -1. [Application introspection and debugging](introspection-and-debugging.html) - 1. [Using the Kubernetes web user interface](ui.html) - 1. [Logging](logging.html) - 1. [Monitoring](monitoring.html) - 1. [Getting into containers via `exec`](getting-into-containers.html) - 1. [Connecting to containers via proxies](connecting-to-applications-proxy.html) - 1. [Connecting to containers via port forwarding](connecting-to-applications-port-forward.html) +1. [Quick start: launch and expose an application](quick-start) +1. [Configuring and launching containers: configuring common container parameters](configuring-containers) +1. [Deploying continuously running applications](deploying-applications) +1. [Connecting applications: exposing applications to clients and users](connecting-applications) +1. [Working with containers in production](production-pods) +1. [Managing deployments](managing-deployments) +1. [Application introspection and debugging](introspection-and-debugging) + 1. [Using the Kubernetes web user interface](ui) + 1. [Logging](logging) + 1. [Monitoring](monitoring) + 1. [Getting into containers via `exec`](getting-into-containers) + 1. [Connecting to containers via proxies](connecting-to-applications-proxy) + 1. [Connecting to containers via port forwarding](connecting-to-applications-port-forward) ## Concept guide -[**Overview**](overview.html) +[**Overview**](overview) : A brief overview of Kubernetes concepts. -[**Cluster**](../admin/README.html) +[**Cluster**](../admin/README) : A cluster is a set of physical or virtual machines and other infrastructure resources used by Kubernetes to run your applications. -[**Node**](../admin/node.html) +[**Node**](../admin/node) : A node is a physical or virtual machine running Kubernetes, onto which pods can be scheduled. -[**Pod**](pods.html) +[**Pod**](pods) : A pod is a co-located group of containers and volumes. -[**Label**](labels.html) +[**Label**](labels) : A label is a key/value pair that is attached to a resource, such as a pod, to convey a user-defined identifying attribute. Labels can be used to organize and to select subsets of resources. [**Selector**](labels.html#label-selectors) : A selector is an expression that matches labels in order to identify related resources, such as which pods are targeted by a load-balanced service. -[**Replication Controller**](replication-controller.html) +[**Replication Controller**](replication-controller) : A replication controller ensures that a specified number of pod replicas are running at any one time. It both allows for easy scaling of replicated systems and handles re-creation of a pod when the machine it is on reboots or otherwise fails. -[**Service**](services.html) +[**Service**](services) : A service defines a set of pods and a means by which to access them, such as single stable IP address and corresponding DNS name. -[**Volume**](volumes.html) +[**Volume**](volumes) : A volume is a directory, possibly with some data in it, which is accessible to a Container as part of its filesystem. Kubernetes volumes build upon [Docker Volumes](https://docs.docker.com/userguide/dockervolumes/), adding provisioning of the volume directory and/or device. -[**Secret**](secrets.html) +[**Secret**](secrets) : A secret stores sensitive data, such as authentication tokens, which can be made available to containers upon request. -[**Name**](identifiers.html) +[**Name**](identifiers) : A user- or client-provided name for a resource. -[**Namespace**](namespaces.html) +[**Namespace**](namespaces) : A namespace is like a prefix to the name of a resource. Namespaces help different projects, teams, or customers to share a cluster, such as by preventing name collisions between unrelated teams. -[**Annotation**](annotations.html) +[**Annotation**](annotations) : A key/value pair that can hold larger (compared to a label), and possibly not human-readable, data, intended to store non-identifying auxiliary data, especially data manipulated by tools and system extensions. Efficient filtering by annotation values is not supported. ## Further reading * API resources - * [Working with resources](working-with-resources.html) + * [Working with resources](working-with-resources) * Pods and containers - * [Pod lifecycle and restart policies](pod-states.html) - * [Lifecycle hooks](container-environment.html) - * [Compute resources, such as cpu and memory](compute-resources.html) - * [Specifying commands and requesting capabilities](containers.html) - * [Downward API: accessing system configuration from a pod](downward-api.html) - * [Images and registries](images.html) - * [Migrating from docker-cli to kubectl](docker-cli-to-kubectl.html) - * [Tips and tricks when working with config](config-best-practices.html) + * [Pod lifecycle and restart policies](pod-states) + * [Lifecycle hooks](container-environment) + * [Compute resources, such as cpu and memory](compute-resources) + * [Specifying commands and requesting capabilities](containers) + * [Downward API: accessing system configuration from a pod](downward-api) + * [Images and registries](images) + * [Migrating from docker-cli to kubectl](docker-cli-to-kubectl) + * [Tips and tricks when working with config](config-best-practices) * [Assign pods to selected nodes](node-selection/) * [Perform a rolling update on a running group of pods](update-demo/) diff --git a/v1.1/docs/user-guide/accessing-the-cluster.md b/v1.1/docs/user-guide/accessing-the-cluster.md index 44f6ac3aae..6b50524d52 100644 --- a/v1.1/docs/user-guide/accessing-the-cluster.md +++ b/v1.1/docs/user-guide/accessing-the-cluster.md @@ -1,282 +1,269 @@ ---- -title: "Accessing Clusters" -section: guides ---- - -## Table of Contents - -{% include pagetoc.html %} - -## Accessing the cluster API - -### Accessing for the first time with kubectl - -When accessing the Kubernetes API for the first time, we suggest using the -Kubernetes CLI, `kubectl`. - -To access a cluster, you need to know the location of the cluster and have credentials -to access it. Typically, this is automatically set-up when you work through -though a [Getting started guide](../getting-started-guides/README.html), -or someone else setup the cluster and provided you with credentials and a location. - -Check the location and credentials that kubectl knows about with this command: - -{% highlight console %} -{% raw %} -$ kubectl config view -{% endraw %} -{% endhighlight %} - -Many of the [examples](../../examples/) provide an introduction to using -kubectl and complete documentation is found in the [kubectl manual](kubectl/kubectl.html). - -### Directly accessing the REST API - -Kubectl handles locating and authenticating to the apiserver. -If you want to directly access the REST API with an http client like -curl or wget, or a browser, there are several ways to locate and authenticate: - - Run kubectl in proxy mode. - - Recommended approach. - - Uses stored apiserver location. - - Verifies identity of apiserver using self-signed cert. No MITM possible. - - Authenticates to apiserver. - - In future, may do intelligent client-side load-balancing and failover. - - Provide the location and credentials directly to the http client. - - Alternate approach. - - Works with some types of client code that are confused by using a proxy. - - Need to import a root cert into your browser to protect against MITM. - -#### Using kubectl proxy - -The following command runs kubectl in a mode where it acts as a reverse proxy. It handles -locating the apiserver and authenticating. -Run it like this: - -{% highlight console %} -{% raw %} -$ kubectl proxy --port=8080 & -{% endraw %} -{% endhighlight %} - -See [kubectl proxy](kubectl/kubectl_proxy.html) for more details. - -Then you can explore the API with curl, wget, or a browser, like so: - -{% highlight console %} -{% raw %} -$ curl http://localhost:8080/api/ -{ - "versions": [ - "v1" - ] -} -{% endraw %} -{% endhighlight %} - -#### Without kubectl proxy - -It is also possible to avoid using kubectl proxy by passing an authentication token -directly to the apiserver, like this: - -{% highlight console %} -{% raw %} -$ APISERVER=$(kubectl config view | grep server | cut -f 2- -d ":" | tr -d " ") -$ TOKEN=$(kubectl config view | grep token | cut -f 2 -d ":" | tr -d " ") -$ curl $APISERVER/api --header "Authorization: Bearer $TOKEN" --insecure -{ - "versions": [ - "v1" - ] -} -{% endraw %} -{% endhighlight %} - -The above example uses the `--insecure` flag. This leaves it subject to MITM -attacks. When kubectl accesses the cluster it uses a stored root certificate -and client certificates to access the server. (These are installed in the -`~/.kube` directory). Since cluster certificates are typically self-signed, it -make take special configuration to get your http client to use root -certificate. - -On some clusters, the apiserver does not require authentication; it may serve -on localhost, or be protected by a firewall. There is not a standard -for this. [Configuring Access to the API](../admin/accessing-the-api.html) -describes how a cluster admin can configure this. Such approaches may conflict -with future high-availability support. - -### Programmatic access to the API - -There are [client libraries](../devel/client-libraries.html) for accessing the API -from several languages. The Kubernetes project-supported -[Go](http://releases.k8s.io/release-1.1/pkg/client/) -client library can use the same [kubeconfig file](kubeconfig-file.html) -as the kubectl CLI does to locate and authenticate to the apiserver. - -See documentation for other libraries for how they authenticate. - -### Accessing the API from a Pod - -When accessing the API from a pod, locating and authenticating -to the api server are somewhat different. - -The recommended way to locate the apiserver within the pod is with -the `kubernetes` DNS name, which resolves to a Service IP which in turn -will be routed to an apiserver. - -The recommended way to authenticate to the apiserver is with a -[service account](service-accounts.html) credential. By kube-system, a pod -is associated with a service account, and a credential (token) for that -service account is placed into the filesystem tree of each container in that pod, -at `/var/run/secrets/kubernetes.io/serviceaccount/token`. - -From within a pod the recommended ways to connect to API are: - - run a kubectl proxy as one of the containers in the pod, or as a background - process within a container. This proxies the - Kubernetes API to the localhost interface of the pod, so that other processes - in any container of the pod can access it. See this [example of using kubectl proxy - in a pod](../../examples/kubectl-container/). - - use the Go client library, and create a client using the `client.NewInCluster()` factory. - This handles locating and authenticating to the apiserver. -In each case, the credentials of the pod are used to communicate securely with the apiserver. - - -## Accessing services running on the cluster - -The previous section was about connecting the Kubernetes API server. This section is about -connecting to other services running on Kubernetes cluster. In Kubernetes, the -[nodes](../admin/node.html), [pods](pods.html) and [services](services.html) all have -their own IPs. In many cases, the node IPs, pod IPs, and some service IPs on a cluster will not be -routable, so they will not be reachable from a machine outside the cluster, -such as your desktop machine. - -### Ways to connect - -You have several options for connecting to nodes, pods and services from outside the cluster: - - Access services through public IPs. - - Use a service with type `NodePort` or `LoadBalancer` to make the service reachable outside - the cluster. See the [services](services.html) and - [kubectl expose](kubectl/kubectl_expose.html) documentation. - - Depending on your cluster environment, this may just expose the service to your corporate network, - or it may expose it to the internet. Think about whether the service being exposed is secure. - Does it do its own authentication? - - Place pods behind services. To access one specific pod from a set of replicas, such as for debugging, - place a unique label on the pod it and create a new service which selects this label. - - In most cases, it should not be necessary for application developer to directly access - nodes via their nodeIPs. - - Access services, nodes, or pods using the Proxy Verb. - - Does apiserver authentication and authorization prior to accessing the remote service. - Use this if the services are not secure enough to expose to the internet, or to gain - access to ports on the node IP, or for debugging. - - Proxies may cause problems for some web applications. - - Only works for HTTP/HTTPS. - - Described [here](#discovering-builtin-services). - - Access from a node or pod in the cluster. - - Run a pod, and then connect to a shell in it using [kubectl exec](kubectl/kubectl_exec.html). - Connect to other nodes, pods, and services from that shell. - - Some clusters may allow you to ssh to a node in the cluster. From there you may be able to - access cluster services. This is a non-standard method, and will work on some clusters but - not others. Browsers and other tools may or may not be installed. Cluster DNS may not work. - -### Discovering builtin services - -Typically, there are several services which are started on a cluster by kube-system. Get a list of these -with the `kubectl cluster-info` command: - -{% highlight console %} -{% raw %} -$ kubectl cluster-info - - Kubernetes master is running at https://104.197.5.247 - elasticsearch-logging is running at https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging - kibana-logging is running at https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/kibana-logging - kube-dns is running at https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/kube-dns - grafana is running at https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana - heapster is running at https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/monitoring-heapster -{% endraw %} -{% endhighlight %} - -This shows the proxy-verb URL for accessing each service. -For example, this cluster has cluster-level logging enabled (using Elasticsearch), which can be reached -at `https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging/` if suitable credentials are passed, or through a kubectl proxy at, for example: -`http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging/`. -(See [above](#accessing-the-cluster-api) for how to pass credentials or use kubectl proxy.) - -#### Manually constructing apiserver proxy URLs - -As mentioned above, you use the `kubectl cluster-info` command to retrieve the service's proxy URL. To create proxy URLs that include service endpoints, suffixes, and parameters, you simply append to the service's proxy URL: -`http://`*`kubernetes_master_address`*`/`*`service_path`*`/`*`service_name`*`/`*`service_endpoint-suffix-parameter`* - - -##### Examples - - * To access the Elasticsearch service endpoint `_search?q=user:kimchy`, you would use: `http://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging/_search?q=user:kimchy` - * To access the Elasticsearch cluster health information `_cluster/health?pretty=true`, you would use: `https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging/_cluster/health?pretty=true` - -{% highlight json %} -{% raw %} - { - "cluster_name" : "kubernetes_logging", - "status" : "yellow", - "timed_out" : false, - "number_of_nodes" : 1, - "number_of_data_nodes" : 1, - "active_primary_shards" : 5, - "active_shards" : 5, - "relocating_shards" : 0, - "initializing_shards" : 0, - "unassigned_shards" : 5 - } -{% endraw %} -{% endhighlight %} - -#### Using web browsers to access services running on the cluster - -You may be able to put an apiserver proxy url into the address bar of a browser. However: - - Web browsers cannot usually pass tokens, so you may need to use basic (password) auth. Apiserver can be configured to accept basic auth, - but your cluster may not be configured to accept basic auth. - - Some web apps may not work, particularly those with client side javascript that construct urls in a - way that is unaware of the proxy path prefix. - -## Requesting redirects - -The redirect capabilities have been deprecated and removed. Please use a proxy (see below) instead. - -## So Many Proxies - -There are several different proxies you may encounter when using Kubernetes: - 1. The [kubectl proxy](#directly-accessing-the-rest-api): - - runs on a user's desktop or in a pod - - proxies from a localhost address to the Kubernetes apiserver - - client to proxy uses HTTP - - proxy to apiserver uses HTTPS - - locates apiserver - - adds authentication headers - 1. The [apiserver proxy](#discovering-builtin-services): - - is a bastion built into the apiserver - - connects a user outside of the cluster to cluster IPs which otherwise might not be reachable - - runs in the apiserver processes - - client to proxy uses HTTPS (or http if apiserver so configured) - - proxy to target may use HTTP or HTTPS as chosen by proxy using available information - - can be used to reach a Node, Pod, or Service - - does load balancing when used to reach a Service - 1. The [kube proxy](services.html#ips-and-vips): - - runs on each node - - proxies UDP and TCP - - does not understand HTTP - - provides load balancing - - is just used to reach services - 1. A Proxy/Load-balancer in front of apiserver(s): - - existence and implementation varies from cluster to cluster (e.g. nginx) - - sits between all clients and one or more apiservers - - acts as load balancer if there are several apiservers. - 1. Cloud Load Balancers on external services: - - are provided by some cloud providers (e.g. AWS ELB, Google Cloud Load Balancer) - - are created automatically when the Kubernetes service has type `LoadBalancer` - - use UDP/TCP only - - implementation varies by cloud provider. - - - -Kubernetes users will typically not need to worry about anything other than the first two types. The cluster admin +--- +title: "Accessing Clusters" +--- + + + +{% include pagetoc.html %} + +## Accessing the cluster API + +### Accessing for the first time with kubectl + +When accessing the Kubernetes API for the first time, we suggest using the +Kubernetes CLI, `kubectl`. + +To access a cluster, you need to know the location of the cluster and have credentials +to access it. Typically, this is automatically set-up when you work through +though a [Getting started guide](../getting-started-guides/README), +or someone else setup the cluster and provided you with credentials and a location. + +Check the location and credentials that kubectl knows about with this command: + +{% highlight console %} +$ kubectl config view +{% endhighlight %} + +Many of the [examples](../../examples/) provide an introduction to using +kubectl and complete documentation is found in the [kubectl manual](kubectl/kubectl). + +### Directly accessing the REST API + +Kubectl handles locating and authenticating to the apiserver. +If you want to directly access the REST API with an http client like +curl or wget, or a browser, there are several ways to locate and authenticate: + - Run kubectl in proxy mode. + - Recommended approach. + - Uses stored apiserver location. + - Verifies identity of apiserver using self-signed cert. No MITM possible. + - Authenticates to apiserver. + - In future, may do intelligent client-side load-balancing and failover. + - Provide the location and credentials directly to the http client. + - Alternate approach. + - Works with some types of client code that are confused by using a proxy. + - Need to import a root cert into your browser to protect against MITM. + +#### Using kubectl proxy + +The following command runs kubectl in a mode where it acts as a reverse proxy. It handles +locating the apiserver and authenticating. +Run it like this: + +{% highlight console %} +$ kubectl proxy --port=8080 & +{% endhighlight %} + +See [kubectl proxy](kubectl/kubectl_proxy) for more details. + +Then you can explore the API with curl, wget, or a browser, like so: + +{% highlight console %} +$ curl http://localhost:8080/api/ +{ + "versions": [ + "v1" + ] +} +{% endhighlight %} + +#### Without kubectl proxy + +It is also possible to avoid using kubectl proxy by passing an authentication token +directly to the apiserver, like this: + +{% highlight console %} +$ APISERVER=$(kubectl config view | grep server | cut -f 2- -d ":" | tr -d " ") +$ TOKEN=$(kubectl config view | grep token | cut -f 2 -d ":" | tr -d " ") +$ curl $APISERVER/api --header "Authorization: Bearer $TOKEN" --insecure +{ + "versions": [ + "v1" + ] +} +{% endhighlight %} + +The above example uses the `--insecure` flag. This leaves it subject to MITM +attacks. When kubectl accesses the cluster it uses a stored root certificate +and client certificates to access the server. (These are installed in the +`~/.kube` directory). Since cluster certificates are typically self-signed, it +make take special configuration to get your http client to use root +certificate. + +On some clusters, the apiserver does not require authentication; it may serve +on localhost, or be protected by a firewall. There is not a standard +for this. [Configuring Access to the API](../admin/accessing-the-api) +describes how a cluster admin can configure this. Such approaches may conflict +with future high-availability support. + +### Programmatic access to the API + +There are [client libraries](../devel/client-libraries) for accessing the API +from several languages. The Kubernetes project-supported +[Go](http://releases.k8s.io/release-1.1/pkg/client/) +client library can use the same [kubeconfig file](kubeconfig-file) +as the kubectl CLI does to locate and authenticate to the apiserver. + +See documentation for other libraries for how they authenticate. + +### Accessing the API from a Pod + +When accessing the API from a pod, locating and authenticating +to the api server are somewhat different. + +The recommended way to locate the apiserver within the pod is with +the `kubernetes` DNS name, which resolves to a Service IP which in turn +will be routed to an apiserver. + +The recommended way to authenticate to the apiserver is with a +[service account](service-accounts) credential. By kube-system, a pod +is associated with a service account, and a credential (token) for that +service account is placed into the filesystem tree of each container in that pod, +at `/var/run/secrets/kubernetes.io/serviceaccount/token`. + +From within a pod the recommended ways to connect to API are: + - run a kubectl proxy as one of the containers in the pod, or as a background + process within a container. This proxies the + Kubernetes API to the localhost interface of the pod, so that other processes + in any container of the pod can access it. See this [example of using kubectl proxy + in a pod](../../examples/kubectl-container/). + - use the Go client library, and create a client using the `client.NewInCluster()` factory. + This handles locating and authenticating to the apiserver. +In each case, the credentials of the pod are used to communicate securely with the apiserver. + + +## Accessing services running on the cluster + +The previous section was about connecting the Kubernetes API server. This section is about +connecting to other services running on Kubernetes cluster. In Kubernetes, the +[nodes](../admin/node), [pods](pods) and [services](services) all have +their own IPs. In many cases, the node IPs, pod IPs, and some service IPs on a cluster will not be +routable, so they will not be reachable from a machine outside the cluster, +such as your desktop machine. + +### Ways to connect + +You have several options for connecting to nodes, pods and services from outside the cluster: + - Access services through public IPs. + - Use a service with type `NodePort` or `LoadBalancer` to make the service reachable outside + the cluster. See the [services](services) and + [kubectl expose](kubectl/kubectl_expose) documentation. + - Depending on your cluster environment, this may just expose the service to your corporate network, + or it may expose it to the internet. Think about whether the service being exposed is secure. + Does it do its own authentication? + - Place pods behind services. To access one specific pod from a set of replicas, such as for debugging, + place a unique label on the pod it and create a new service which selects this label. + - In most cases, it should not be necessary for application developer to directly access + nodes via their nodeIPs. + - Access services, nodes, or pods using the Proxy Verb. + - Does apiserver authentication and authorization prior to accessing the remote service. + Use this if the services are not secure enough to expose to the internet, or to gain + access to ports on the node IP, or for debugging. + - Proxies may cause problems for some web applications. + - Only works for HTTP/HTTPS. + - Described [here](#discovering-builtin-services). + - Access from a node or pod in the cluster. + - Run a pod, and then connect to a shell in it using [kubectl exec](kubectl/kubectl_exec). + Connect to other nodes, pods, and services from that shell. + - Some clusters may allow you to ssh to a node in the cluster. From there you may be able to + access cluster services. This is a non-standard method, and will work on some clusters but + not others. Browsers and other tools may or may not be installed. Cluster DNS may not work. + +### Discovering builtin services + +Typically, there are several services which are started on a cluster by kube-system. Get a list of these +with the `kubectl cluster-info` command: + +{% highlight console %} +$ kubectl cluster-info + + Kubernetes master is running at https://104.197.5.247 + elasticsearch-logging is running at https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging + kibana-logging is running at https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/kibana-logging + kube-dns is running at https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/kube-dns + grafana is running at https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana + heapster is running at https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/monitoring-heapster +{% endhighlight %} + +This shows the proxy-verb URL for accessing each service. +For example, this cluster has cluster-level logging enabled (using Elasticsearch), which can be reached +at `https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging/` if suitable credentials are passed, or through a kubectl proxy at, for example: +`http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging/`. +(See [above](#accessing-the-cluster-api) for how to pass credentials or use kubectl proxy.) + +#### Manually constructing apiserver proxy URLs + +As mentioned above, you use the `kubectl cluster-info` command to retrieve the service's proxy URL. To create proxy URLs that include service endpoints, suffixes, and parameters, you simply append to the service's proxy URL: +`http://`*`kubernetes_master_address`*`/`*`service_path`*`/`*`service_name`*`/`*`service_endpoint-suffix-parameter`* + + +##### Examples + + * To access the Elasticsearch service endpoint `_search?q=user:kimchy`, you would use: `http://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging/_search?q=user:kimchy` + * To access the Elasticsearch cluster health information `_cluster/health?pretty=true`, you would use: `https://104.197.5.247/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging/_cluster/health?pretty=true` + +{% highlight json %} + { + "cluster_name" : "kubernetes_logging", + "status" : "yellow", + "timed_out" : false, + "number_of_nodes" : 1, + "number_of_data_nodes" : 1, + "active_primary_shards" : 5, + "active_shards" : 5, + "relocating_shards" : 0, + "initializing_shards" : 0, + "unassigned_shards" : 5 + } +{% endhighlight %} + +#### Using web browsers to access services running on the cluster + +You may be able to put an apiserver proxy url into the address bar of a browser. However: + - Web browsers cannot usually pass tokens, so you may need to use basic (password) auth. Apiserver can be configured to accept basic auth, + but your cluster may not be configured to accept basic auth. + - Some web apps may not work, particularly those with client side javascript that construct urls in a + way that is unaware of the proxy path prefix. + +## Requesting redirects + +The redirect capabilities have been deprecated and removed. Please use a proxy (see below) instead. + +## So Many Proxies + +There are several different proxies you may encounter when using Kubernetes: + 1. The [kubectl proxy](#directly-accessing-the-rest-api): + - runs on a user's desktop or in a pod + - proxies from a localhost address to the Kubernetes apiserver + - client to proxy uses HTTP + - proxy to apiserver uses HTTPS + - locates apiserver + - adds authentication headers + 1. The [apiserver proxy](#discovering-builtin-services): + - is a bastion built into the apiserver + - connects a user outside of the cluster to cluster IPs which otherwise might not be reachable + - runs in the apiserver processes + - client to proxy uses HTTPS (or http if apiserver so configured) + - proxy to target may use HTTP or HTTPS as chosen by proxy using available information + - can be used to reach a Node, Pod, or Service + - does load balancing when used to reach a Service + 1. The [kube proxy](services.html#ips-and-vips): + - runs on each node + - proxies UDP and TCP + - does not understand HTTP + - provides load balancing + - is just used to reach services + 1. A Proxy/Load-balancer in front of apiserver(s): + - existence and implementation varies from cluster to cluster (e.g. nginx) + - sits between all clients and one or more apiservers + - acts as load balancer if there are several apiservers. + 1. Cloud Load Balancers on external services: + - are provided by some cloud providers (e.g. AWS ELB, Google Cloud Load Balancer) + - are created automatically when the Kubernetes service has type `LoadBalancer` + - use UDP/TCP only + - implementation varies by cloud provider. + + + +Kubernetes users will typically not need to worry about anything other than the first two types. The cluster admin will typically ensure that the latter types are setup correctly. \ No newline at end of file diff --git a/v1.1/docs/user-guide/annotations.md b/v1.1/docs/user-guide/annotations.md index 573bd6e1bb..84898d7e0b 100644 --- a/v1.1/docs/user-guide/annotations.md +++ b/v1.1/docs/user-guide/annotations.md @@ -1,23 +1,19 @@ --- title: "Annotations" --- - - -# Annotations - -We have [labels](labels.html) for identifying metadata. +We have [labels](labels) for identifying metadata. It is also useful to be able to attach arbitrary non-identifying metadata, for retrieval by API clients such as tools, libraries, etc. This information may be large, may be structured or unstructured, may include characters not permitted by labels, etc. Such information would not be used for object selection and therefore doesn't belong in labels. Like labels, annotations are key-value maps. {% highlight json %} -{% raw %} + "annotations": { "key1" : "value1", "key2" : "value2" } -{% endraw %} + {% endhighlight %} Possible information that could be recorded in annotations: diff --git a/v1.1/docs/user-guide/application-troubleshooting.md b/v1.1/docs/user-guide/application-troubleshooting.md index a8146831c3..97be9958c7 100644 --- a/v1.1/docs/user-guide/application-troubleshooting.md +++ b/v1.1/docs/user-guide/application-troubleshooting.md @@ -1,32 +1,11 @@ --- title: "Application Troubleshooting" --- - - -# Application Troubleshooting - This guide is to help users debug applications that are deployed into Kubernetes and not behaving correctly. This is *not* a guide for people who want to debug their cluster. For that you should check out -[this guide](../admin/cluster-troubleshooting.html) +[this guide](../admin/cluster-troubleshooting) -**Table of Contents** - - -- [Application Troubleshooting](#application-troubleshooting) - - [FAQ](#faq) - - [Diagnosing the problem](#diagnosing-the-problem) - - [Debugging Pods](#debugging-pods) - - [My pod stays pending](#my-pod-stays-pending) - - [My pod stays waiting](#my-pod-stays-waiting) - - [My pod is crashing or otherwise unhealthy](#my-pod-is-crashing-or-otherwise-unhealthy) - - [My pod is running but not doing what I told it to do](#my-pod-is-running-but-not-doing-what-i-told-it-to-do) - - [Debugging Replication Controllers](#debugging-replication-controllers) - - [Debugging Services](#debugging-services) - - [My service is missing endpoints](#my-service-is-missing-endpoints) - - [Network traffic is not forwarded](#network-traffic-is-not-forwarded) - - [More information](#more-information) - - +{% include pagetoc.html %} ## FAQ @@ -45,9 +24,9 @@ your Service? The first step in debugging a Pod is taking a look at it. Check the current state of the Pod and recent events with the following command: {% highlight console %} -{% raw %} + $ kubectl describe pods ${POD_NAME} -{% endraw %} + {% endhighlight %} Look at the state of the containers in the pod. Are they all `Running`? Have there been recent restarts? @@ -83,25 +62,25 @@ First, take a look at the logs of the current container: {% highlight console %} -{% raw %} + $ kubectl logs ${POD_NAME} ${CONTAINER_NAME} -{% endraw %} + {% endhighlight %} If your container has previously crashed, you can access the previous container's crash log with: {% highlight console %} -{% raw %} + $ kubectl logs --previous ${POD_NAME} ${CONTAINER_NAME} -{% endraw %} + {% endhighlight %} Alternately, you can run commands inside that container with `exec`: {% highlight console %} -{% raw %} + $ kubectl exec ${POD_NAME} -c ${CONTAINER_NAME} -- ${CMD} ${ARG1} ${ARG2} ... ${ARGN} -{% endraw %} + {% endhighlight %} Note that `-c ${CONTAINER_NAME}` is optional and can be omitted for Pods that only contain a single container. @@ -109,9 +88,9 @@ Note that `-c ${CONTAINER_NAME}` is optional and can be omitted for Pods that on As an example, to look at the logs from a running Cassandra pod, you might run {% highlight console %} -{% raw %} + $ kubectl exec cassandra -- cat /var/log/cassandra/system.log -{% endraw %} + {% endhighlight %} @@ -133,11 +112,11 @@ For example, run `kubectl create --validate -f mypod.yaml`. If you misspelled `command` as `commnd` then will give an error like this: ``` -{% raw %} + I0805 10:43:25.129850 46757 schema.go:126] unknown field: commnd I0805 10:43:25.129973 46757 schema.go:129] this may be a false alarm, see https://github.com/kubernetes/kubernetes/issues/6842 pods/mypod -{% endraw %} + ``` @@ -169,9 +148,9 @@ First, verify that there are endpoints for the service. For every Service object You can view this resource with: {% highlight console %} -{% raw %} + $ kubectl get endpoints ${SERVICE_NAME} -{% endraw %} + {% endhighlight %} Make sure that the endpoints match up with the number of containers that you expect to be a member of your service. @@ -184,21 +163,21 @@ If you are missing endpoints, try listing pods using the labels that Service use a Service where the labels are: {% highlight yaml %} -{% raw %} + ... spec: - selector: name: nginx type: frontend -{% endraw %} + {% endhighlight %} You can use: {% highlight console %} -{% raw %} + $ kubectl get pods --selector=name=nginx,type=frontend -{% endraw %} + {% endhighlight %} to list pods that match this selector. Verify that the list matches the Pods that you expect to provide your Service. @@ -222,9 +201,9 @@ check: #### More information -If none of the above solves your problem, follow the instructions in [Debugging Service document](debugging-services.html) to make sure that your `Service` is running, has `Endpoints`, and your `Pods` are actually serving; you have DNS working, iptables rules installed, and kube-proxy does not seem to be misbehaving. +If none of the above solves your problem, follow the instructions in [Debugging Service document](debugging-services) to make sure that your `Service` is running, has `Endpoints`, and your `Pods` are actually serving; you have DNS working, iptables rules installed, and kube-proxy does not seem to be misbehaving. -You may also visit [troubleshooting document](../troubleshooting.html) for more information. +You may also visit [troubleshooting document](../troubleshooting) for more information. diff --git a/v1.1/docs/user-guide/compute-resources.md b/v1.1/docs/user-guide/compute-resources.md index 32fc330af2..83756c2bd7 100644 --- a/v1.1/docs/user-guide/compute-resources.md +++ b/v1.1/docs/user-guide/compute-resources.md @@ -1,39 +1,22 @@ --- title: "Compute Resources" --- +{% include pagetoc.html %} - -# Compute Resources - -**Table of Contents** - - -- [Compute Resources](#compute-resources) - - [Resource Requests and Limits of Pod and Container](#resource-requests-and-limits-of-pod-and-container) - - [How Pods with Resource Requests are Scheduled](#how-pods-with-resource-requests-are-scheduled) - - [How Pods with Resource Limits are Run](#how-pods-with-resource-limits-are-run) - - [Monitoring Compute Resource Usage](#monitoring-compute-resource-usage) - - [Troubleshooting](#troubleshooting) - - [My pods are pending with event message failedScheduling](#my-pods-are-pending-with-event-message-failedscheduling) - - [My container is terminated](#my-container-is-terminated) - - [Planned Improvements](#planned-improvements) - - - -When specifying a [pod](pods.html), you can optionally specify how much CPU and memory (RAM) each +When specifying a [pod](pods), you can optionally specify how much CPU and memory (RAM) each container needs. When containers have their resource requests specified, the scheduler is able to make better decisions about which nodes to place pods on; and when containers have their limits specified, contention for resources on a node can be handled in a specified manner. For more details about the difference between requests and limits, please refer to -[Resource QoS](../proposals/resource-qos.html). +[Resource QoS](../proposals/resource-qos). *CPU* and *memory* are each a *resource type*. A resource type has a base unit. CPU is specified in units of cores. Memory is specified in units of bytes. CPU and RAM are collectively referred to as *compute resources*, or just *resources*. Compute resources are measureable quantities which can be requested, allocated, and consumed. They are -distinct from [API resources](working-with-resources.html). API resources, such as pods and -[services](services.html) are objects that can be written to and retrieved from the Kubernetes API +distinct from [API resources](working-with-resources). API resources, such as pods and +[services](services) are objects that can be written to and retrieved from the Kubernetes API server. ## Resource Requests and Limits of Pod and Container @@ -59,7 +42,7 @@ be said to have a request of 0.5 core and 128 MiB of memory and a limit of 1 cor memory. {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: @@ -84,7 +67,7 @@ spec: limits: memory: "128Mi" cpu: "500m" -{% endraw %} + {% endhighlight %} ## How Pods with Resource Requests are Scheduled @@ -140,13 +123,13 @@ until a place can be found. An event will be produced each time the scheduler place for the pod, like this: {% highlight console %} -{% raw %} + $ kubectl describe pod frontend | grep -A 3 Events Events: FirstSeen LastSeen Count From Subobject PathReason Message 36s 5s 6 {scheduler } FailedScheduling Failed for reason PodExceedsFreeCPU and possibly others -{% endraw %} + {% endhighlight %} In the case shown above, the pod "frontend" fails to be scheduled due to insufficient @@ -162,7 +145,7 @@ You can check node capacities and amounts allocated with the `kubectl describe n For example: {% highlight console %} -{% raw %} + $ kubectl describe nodes gke-cluster-4-386701dd-node-ww4p Name: gke-cluster-4-386701dd-node-ww4p [ ... lines removed for clarity ...] @@ -184,7 +167,7 @@ TotalResourceLimits: CPU(milliCPU): 910 (91% of total) Memory(bytes): 2485125120 (59% of total) [ ... lines removed for clarity ...] -{% endraw %} + {% endhighlight %} Here you can see from the `Allocated resources` section that that a pod which ask for more than @@ -192,7 +175,7 @@ Here you can see from the `Allocated resources` section that that a pod which as Looking at the `Pods` section, you can see which pods are taking up space on the node. -The [resource quota](../admin/resource-quota.html) feature can be configured +The [resource quota](../admin/resource-quota) feature can be configured to limit the total amount of resources that can be consumed. If used in conjunction with namespaces, it can prevent one team from hogging all the resources. @@ -202,7 +185,7 @@ Your container may be terminated because it's resource-starved. To check if a co on the pod you are interested in: {% highlight console %} -{% raw %} + [12:54:41] $ ./cluster/kubectl.sh describe pod simmemleak-hra99 Name: simmemleak-hra99 Namespace: default @@ -238,7 +221,7 @@ Events: Tue, 07 Jul 2015 12:53:51 -0700 Tue, 07 Jul 2015 12:53:51 -0700 1 {kubelet kubernetes-minion-tf0f} implicitly required container POD created Created with docker id 6a41280f516d Tue, 07 Jul 2015 12:53:51 -0700 Tue, 07 Jul 2015 12:53:51 -0700 1 {kubelet kubernetes-minion-tf0f} implicitly required container POD started Started with docker id 6a41280f516d Tue, 07 Jul 2015 12:53:51 -0700 Tue, 07 Jul 2015 12:53:51 -0700 1 {kubelet kubernetes-minion-tf0f} spec.containers{simmemleak} created Created with docker id 87348f12526a -{% endraw %} + {% endhighlight %} The `Restart Count: 5` indicates that the `simmemleak` container in this pod was terminated and restarted 5 times. @@ -246,11 +229,11 @@ The `Restart Count: 5` indicates that the `simmemleak` container in this pod wa You can call `get pod` with the `-o go-template=...` option to fetch the status of previously terminated containers: {% highlight console %} -{% raw %} + [13:59:01] $ ./cluster/kubectl.sh get pod -o go-template='{{range.status.containerStatuses}}{{"Container Name: "}}{{.name}}{{"\r\nLastState: "}}{{.lastState}}{{end}}' simmemleak-60xbc Container Name: simmemleak LastState: map[terminated:map[exitCode:137 reason:OOM Killed startedAt:2015-07-07T20:58:43Z finishedAt:2015-07-07T20:58:43Z containerID:docker://0e4095bba1feccdfe7ef9fb6ebffe972b4b14285d5acdec6f0d3ae8a22fad8b2]][13:59:03] clusterScaleDoc ~/go/src/github.com/kubernetes/kubernetes $ -{% endraw %} + {% endhighlight %} We can see that this container was terminated because `reason:OOM Killed`, where *OOM* stands for Out Of Memory. diff --git a/v1.1/docs/user-guide/config-best-practices.md b/v1.1/docs/user-guide/config-best-practices.md index 23a5a6f24b..30ddeade8e 100644 --- a/v1.1/docs/user-guide/config-best-practices.md +++ b/v1.1/docs/user-guide/config-best-practices.md @@ -1,10 +1,6 @@ --- title: "Tips and tricks when working with config" --- - - -# Tips and tricks when working with config - This document is meant to highlight and consolidate in one place configuration best practices that are introduced throughout the user-guide and getting-started documentation and examples. This is a living document so if you think of something that is not on this list but might be useful to others, please don't hesitate to file an issue or submit a PR. 1. When writing configuration, use the latest stable API version (currently v1). @@ -13,16 +9,16 @@ This document is meant to highlight and consolidate in one place configuration b 1. Group related objects together in a single file. This is often better than separate files. 1. Use `kubectl create -f ` where possible. This looks for config objects in all `.yaml`, `.yml`, and `.json` files in `` and passes them to create. 1. Create a service before corresponding replication controllers so that the scheduler can spread the pods comprising the service. You can also create the replication controller without specifying replicas, create the service, then scale up the replication controller, which may work better in an example using progressive disclosure and may have benefits in real scenarios also, such as ensuring one replica works before creating lots of them) -1. Don't use `hostPort` unless absolutely necessary (e.g., for a node daemon) as it will prevent certain scheduling configurations due to port conflicts. Use the apiserver proxying or port forwarding for debug/admin access, or a service for external service access. If you need to expose a pod's port on the host machine, consider using a [NodePort](services.html#type--loadbalancer) service before resorting to `hostPort`. If you only need access to the port for debugging purposes, you can also use the [kubectl proxy and apiserver proxy](connecting-to-applications-proxy.html) or [kubectl port-forward](connecting-to-applications-port-forward.html). +1. Don't use `hostPort` unless absolutely necessary (e.g., for a node daemon) as it will prevent certain scheduling configurations due to port conflicts. Use the apiserver proxying or port forwarding for debug/admin access, or a service for external service access. If you need to expose a pod's port on the host machine, consider using a [NodePort](services.html#type--loadbalancer) service before resorting to `hostPort`. If you only need access to the port for debugging purposes, you can also use the [kubectl proxy and apiserver proxy](connecting-to-applications-proxy) or [kubectl port-forward](connecting-to-applications-port-forward). 1. Don't use `hostNetwork` for the same reasons as `hostPort`. 1. Don't specify default values unnecessarily, to simplify and minimize configs. For example, omit the selector and labels in ReplicationController if you want them to be the same as the labels in its podTemplate, since those fields are populated from the podTemplate labels by default. 1. Instead of attaching one label to a set of pods to represent a service (e.g., `service: myservice`) and another to represent the replication controller managing the pods (e.g., `controller: mycontroller`), attach labels that identify semantic attributes of your application or deployment and select the appropriate subsets in your service and replication controller, such as `{ app: myapp, tier: frontend, deployment: v3 }`. A service can be made to span multiple deployments, such as across rolling updates, by simply omitting release-specific labels from its selector, rather than updating a service's selector to match the replication controller's selector fully. 1. Use kubectl bulk operations (via files and/or labels) for get and delete. See [label selectors](labels.html#label-selectors) and [using labels effectively](managing-deployments.html#using-labels-effectively). -1. Use kubectl run and expose to quickly create and expose single container replication controllers. See the [quick start guide](quick-start.html) for an example. +1. Use kubectl run and expose to quickly create and expose single container replication controllers. See the [quick start guide](quick-start) for an example. 1. Use headless services for easy service discovery when you don't need kube-proxy load balancing. See [headless services](services.html#headless-services). 1. Use kubectl delete rather than stop. Delete has a superset of the functionality of stop and stop is deprecated. 1. If there is a viable alternative to naked pods (i.e. pods not bound to a controller), go with the alternative. Controllers are almost always preferable to creating pods (except for some `restartPolicy: Never` scenarios). A minimal Job is coming. See [#1624](http://issue.k8s.io/1624). Naked pods will not be rescheduled in the event of node failure. -1. Put a version number or hash as a suffix to the name and in a label on a replication controller to facilitate rolling update, as we do for [--image](kubectl/kubectl_rolling-update.html). This is necessary because rolling-update actually creates a new controller as opposed to modifying the existing controller. This does not play well with version agnostic controller names. +1. Put a version number or hash as a suffix to the name and in a label on a replication controller to facilitate rolling update, as we do for [--image](kubectl/kubectl_rolling-update). This is necessary because rolling-update actually creates a new controller as opposed to modifying the existing controller. This does not play well with version agnostic controller names. 1. Put an object description in an annotation to allow better introspection. diff --git a/v1.1/docs/user-guide/configuring-containers.md b/v1.1/docs/user-guide/configuring-containers.md index fc06874715..82ce71c77a 100644 --- a/v1.1/docs/user-guide/configuring-containers.md +++ b/v1.1/docs/user-guide/configuring-containers.md @@ -1,92 +1,75 @@ --- title: "Kubernetes User Guide: Managing Applications: Configuring and launching containers" --- - - -# Kubernetes User Guide: Managing Applications: Configuring and launching containers - -**Table of Contents** - - -- [Kubernetes User Guide: Managing Applications: Configuring and launching containers](#kubernetes-user-guide-managing-applications-configuring-and-launching-containers) - - [Configuration in Kubernetes](#configuration-in-kubernetes) - - [Launching a container using a configuration file](#launching-a-container-using-a-configuration-file) - - [Validating configuration](#validating-configuration) - - [Environment variables and variable expansion](#environment-variables-and-variable-expansion) - - [Viewing pod status](#viewing-pod-status) - - [Viewing pod output](#viewing-pod-output) - - [Deleting pods](#deleting-pods) - - [What's next?](#whats-next) - - +{% include pagetoc.html %} ## Configuration in Kubernetes -In addition to the imperative-style commands, such as `kubectl run` and `kubectl expose`, described [elsewhere](quick-start.html), Kubernetes supports declarative configuration. Often times, configuration files are preferable to imperative commands, since they can be checked into version control and changes to the files can be code reviewed, which is especially important for more complex configurations, producing a more robust, reliable and archival system. +In addition to the imperative-style commands, such as `kubectl run` and `kubectl expose`, described [elsewhere](quick-start), Kubernetes supports declarative configuration. Often times, configuration files are preferable to imperative commands, since they can be checked into version control and changes to the files can be code reviewed, which is especially important for more complex configurations, producing a more robust, reliable and archival system. -In the declarative style, all configuration is stored in YAML or JSON configuration files using Kubernetes's API resource schemas as the configuration schemas. `kubectl` can create, update, delete, and get API resources. The `apiVersion` (currently “v1â€?), resource `kind`, and resource `name` are used by `kubectl` to construct the appropriate API path to invoke for the specified operation. +In the declarative style, all configuration is stored in YAML or JSON configuration files using Kubernetes's API resource schemas as the configuration schemas. `kubectl` can create, update, delete, and get API resources. The `apiVersion` (currently 'v1'?), resource `kind`, and resource `name` are used by `kubectl` to construct the appropriate API path to invoke for the specified operation. ## Launching a container using a configuration file -Kubernetes executes containers in [*Pods*](pods.html). A pod containing a simple Hello World container can be specified in YAML as follows: +Kubernetes executes containers in [*Pods*](pods). A pod containing a simple Hello World container can be specified in YAML as follows: {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: name: hello-world -spec: # specification of the pod’s contents +spec: # specification of the pod's contents restartPolicy: Never containers: - name: hello image: "ubuntu:14.04" - command: ["/bin/echo","helloâ€?,â€?world"] -{% endraw %} + command: ["/bin/echo","hello'?,'?world"] + {% endhighlight %} The value of `metadata.name`, `hello-world`, will be the name of the pod resource created, and must be unique within the cluster, whereas `containers[0].name` is just a nickname for the container within that pod. `image` is the name of the Docker image, which Kubernetes expects to be able to pull from a registry, the [Docker Hub](https://registry.hub.docker.com/) by default. `restartPolicy: Never` indicates that we just want to run the container once and then terminate the pod. -The [`command`](containers.html#containers-and-commands) overrides the Docker container’s `Entrypoint`. Command arguments (corresponding to Docker’s `Cmd`) may be specified using `args`, as follows: +The [`command`](containers.html#containers-and-commands) overrides the Docker container's `Entrypoint`. Command arguments (corresponding to Docker's `Cmd`) may be specified using `args`, as follows: {% highlight yaml %} -{% raw %} + command: ["/bin/echo"] args: ["hello","world"] -{% endraw %} + {% endhighlight %} This pod can be created using the `create` command: {% highlight console %} -{% raw %} + $ kubectl create -f ./hello-world.yaml pods/hello-world -{% endraw %} + {% endhighlight %} `kubectl` prints the resource type and name of the resource created when successful. ## Validating configuration -If you’re not sure you specified the resource correctly, you can ask `kubectl` to validate it for you: +If you're not sure you specified the resource correctly, you can ask `kubectl` to validate it for you: {% highlight console %} -{% raw %} + $ kubectl create -f ./hello-world.yaml --validate -{% endraw %} + {% endhighlight %} -Let’s say you specified `entrypoint` instead of `command`. You’d see output as follows: +Let's say you specified `entrypoint` instead of `command`. You'd see output as follows: {% highlight console %} -{% raw %} + I0709 06:33:05.600829 14160 schema.go:126] unknown field: entrypoint I0709 06:33:05.600988 14160 schema.go:129] this may be a false alarm, see http://issue.k8s.io/6842 pods/hello-world -{% endraw %} + {% endhighlight %} `kubectl create --validate` currently warns about problems it detects, but creates the resource anyway, unless a required field is absent or a field value is invalid. Unknown API fields are ignored, so be careful. This pod was created, but with no `command`, which is an optional field, since the image may specify an `Entrypoint`. @@ -99,12 +82,12 @@ to see the list of valid fields. Kubernetes [does not automatically run commands in a shell](https://github.com/kubernetes/kubernetes/wiki/User-FAQ#use-of-environment-variables-on-the-command-line) (not all images contain shells). If you would like to run your command in a shell, such as to expand environment variables (specified using `env`), you could do the following: {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: name: hello-world -spec: # specification of the pod’s contents +spec: # specification of the pod's contents restartPolicy: Never containers: - name: hello @@ -114,42 +97,42 @@ spec: # specification of the pod’s contents value: "hello world" command: ["/bin/sh","-c"] args: ["/bin/echo \"${MESSAGE}\""] -{% endraw %} + {% endhighlight %} -However, a shell isn’t necessary just to expand environment variables. Kubernetes will do it for you if you use [`$(ENVVAR)` syntax](../../docs/design/expansion.html): +However, a shell isn't necessary just to expand environment variables. Kubernetes will do it for you if you use [`$(ENVVAR)` syntax](/{{page.version}}/docs/design/expansion): {% highlight yaml %} -{% raw %} + command: ["/bin/echo"] args: ["$(MESSAGE)"] -{% endraw %} + {% endhighlight %} ## Viewing pod status You can see the pod you created (actually all of your cluster's pods) using the `get` command. -If you’re quick, it will look as follows: +If you're quick, it will look as follows: {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY STATUS RESTARTS AGE hello-world 0/1 Pending 0 0s -{% endraw %} + {% endhighlight %} -Initially, a newly created pod is unscheduled -- no node has been selected to run it. Scheduling happens after creation, but is fast, so you normally shouldn’t see pods in an unscheduled state unless there’s a problem. +Initially, a newly created pod is unscheduled -- no node has been selected to run it. Scheduling happens after creation, but is fast, so you normally shouldn't see pods in an unscheduled state unless there's a problem. -After the pod has been scheduled, the image may need to be pulled to the node on which it was scheduled, if it hadn’t been pulled already. After a few seconds, you should see the container running: +After the pod has been scheduled, the image may need to be pulled to the node on which it was scheduled, if it hadn't been pulled already. After a few seconds, you should see the container running: {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY STATUS RESTARTS AGE hello-world 1/1 Running 0 5s -{% endraw %} + {% endhighlight %} The `READY` column shows how many containers in the pod are running. @@ -157,11 +140,11 @@ The `READY` column shows how many containers in the pod are running. Almost immediately after it starts running, this command will terminate. `kubectl` shows that the container is no longer running and displays the exit status: {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY STATUS RESTARTS AGE hello-world 0/1 ExitCode:0 0 15s -{% endraw %} + {% endhighlight %} ## Viewing pod output @@ -169,21 +152,21 @@ hello-world 0/1 ExitCode:0 0 15s You probably want to see the output of the command you ran. As with [`docker logs`](https://docs.docker.com/userguide/usingdocker/), `kubectl logs` will show you the output: {% highlight console %} -{% raw %} + $ kubectl logs hello-world hello world -{% endraw %} + {% endhighlight %} ## Deleting pods -When you’re done looking at the output, you should delete the pod: +When you're done looking at the output, you should delete the pod: {% highlight console %} -{% raw %} + $ kubectl delete pod hello-world pods/hello-world -{% endraw %} + {% endhighlight %} As with `create`, `kubectl` prints the resource type and name of the resource deleted when successful. @@ -191,19 +174,19 @@ As with `create`, `kubectl` prints the resource type and name of the resource de You can also use the resource/name format to specify the pod: {% highlight console %} -{% raw %} + $ kubectl delete pods/hello-world pods/hello-world -{% endraw %} + {% endhighlight %} -Terminated pods aren’t currently automatically deleted, so that you can observe their final status, so be sure to clean up your dead pods. +Terminated pods aren't currently automatically deleted, so that you can observe their final status, so be sure to clean up your dead pods. On the other hand, containers and their logs are eventually deleted automatically in order to free up disk space on the nodes. ## What's next? -[Learn about deploying continuously running applications.](deploying-applications.html) +[Learn about deploying continuously running applications.](deploying-applications) diff --git a/v1.1/docs/user-guide/connecting-applications.md b/v1.1/docs/user-guide/connecting-applications.md index 2909569082..a05bf786c5 100644 --- a/v1.1/docs/user-guide/connecting-applications.md +++ b/v1.1/docs/user-guide/connecting-applications.md @@ -1,25 +1,7 @@ --- title: "Kubernetes User Guide: Managing Applications: Connecting applications" --- - - -# Kubernetes User Guide: Managing Applications: Connecting applications - -**Table of Contents** - - -- [Kubernetes User Guide: Managing Applications: Connecting applications](#kubernetes-user-guide-managing-applications-connecting-applications) -- [The Kubernetes model for connecting containers](#the-kubernetes-model-for-connecting-containers) - - [Exposing pods to the cluster](#exposing-pods-to-the-cluster) - - [Creating a Service](#creating-a-service) - - [Accessing the Service](#accessing-the-service) - - [Environment Variables](#environment-variables) - - [DNS](#dns) - - [Securing the Service](#securing-the-service) - - [Exposing the Service](#exposing-the-service) - - [What's next?](#whats-next) - - +{% include pagetoc.html %} # The Kubernetes model for connecting containers @@ -27,16 +9,16 @@ Now that you have a continuously running, replicated application you can expose By default, Docker uses host-private networking, so containers can talk to other containers only if they are on the same machine. In order for Docker containers to communicate across nodes, they must be allocated ports on the machine's own IP address, which are then forwarded or proxied to the containers. This obviously means that containers must either coordinate which ports they use very carefully or else be allocated ports dynamically. -Coordinating ports across multiple developers is very difficult to do at scale and exposes users to cluster-level issues outside of their control. Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. We give every pod its own cluster-private-IP address so you do not need to explicitly create links between pods or mapping container ports to host ports. This means that containers within a Pod can all reach each other’s ports on localhost, and all pods in a cluster can see each other without NAT. The rest of this document will elaborate on how you can run reliable services on such a networking model. +Coordinating ports across multiple developers is very difficult to do at scale and exposes users to cluster-level issues outside of their control. Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. We give every pod its own cluster-private-IP address so you do not need to explicitly create links between pods or mapping container ports to host ports. This means that containers within a Pod can all reach each other's ports on localhost, and all pods in a cluster can see each other without NAT. The rest of this document will elaborate on how you can run reliable services on such a networking model. -This guide uses a simple nginx server to demonstrate proof of concept. The same principles are embodied in a more complete [Jenkins CI application](http://blog.kubernetes.io/2015/07/strong-simple-ssl-for-kubernetes.html). +This guide uses a simple nginx server to demonstrate proof of concept. The same principles are embodied in a more complete [Jenkins CI application](http://blog.kubernetes.io/2015/07/strong-simple-ssl-for-kubernetes). ## Exposing pods to the cluster We did this in a previous example, but lets do it once again and focus on the networking perspective. Create an nginx pod, and note that it has a container port specification: {% highlight yaml %} -{% raw %} + $ cat nginxrc.yaml apiVersion: v1 kind: ReplicationController @@ -54,33 +36,33 @@ spec: image: nginx ports: - containerPort: 80 -{% endraw %} + {% endhighlight %} This makes it accessible from any node in your cluster. Check the nodes the pod is running on: {% highlight console %} -{% raw %} + $ kubectl create -f ./nginxrc.yaml $ kubectl get pods -l app=nginx -o wide my-nginx-6isf4 1/1 Running 0 2h e2e-test-beeps-minion-93ly my-nginx-t26zt 1/1 Running 0 2h e2e-test-beeps-minion-93ly -{% endraw %} + {% endhighlight %} Check your pods' IPs: {% highlight console %} -{% raw %} + $ kubectl get pods -l app=nginx -o json | grep podIP "podIP": "10.245.0.15", "podIP": "10.245.0.14", -{% endraw %} + {% endhighlight %} You should be able to ssh into any node in your cluster and curl both IPs. Note that the containers are *not* using port 80 on the node, nor are there any special NAT rules to route traffic to the pod. This means you can run multiple nginx pods on the same node all using the same containerPort and access them from any other pod or node in your cluster using IP. Like Docker, ports can still be published to the host node's interface(s), but the need for this is radically diminished because of the networking model. -You can read more about [how we achieve this](../admin/networking.html#how-to-achieve-this) if you’re curious. +You can read more about [how we achieve this](../admin/networking.html#how-to-achieve-this) if you're curious. ## Creating a Service @@ -91,7 +73,7 @@ A Kubernetes Service is an abstraction which defines a logical set of Pods runni You can create a Service for your 2 nginx replicas with the following yaml: {% highlight yaml %} -{% raw %} + $ cat nginxsvc.yaml apiVersion: v1 kind: Service @@ -105,25 +87,25 @@ spec: protocol: TCP selector: app: nginx -{% endraw %} + {% endhighlight %} This specification will create a Service which targets TCP port 80 on any Pod with the `app=nginx` label, and expose it on an abstracted Service port (`targetPort`: is the port the container accepts traffic on, `port`: is the abstracted Service port, which can be any port other pods use to access the Service). View [service API object](http://kubernetes.io/v1.1/docs/api-reference/v1/definitions.html#_v1_service) to see the list of supported fields in service definition. Check your Service: {% highlight console %} -{% raw %} + $ kubectl get svc NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE kubernetes 10.179.240.1 443/TCP 8d nginxsvc 10.179.252.126 122.222.183.144 80/TCP,81/TCP,82/TCP run=nginx2 11m -{% endraw %} + {% endhighlight %} -As mentioned previously, a Service is backed by a group of pods. These pods are exposed through `endpoints`. The Service's selector will be evaluated continuously and the results will be POSTed to an Endpoints object also named `nginxsvc`. When a pod dies, it is automatically removed from the endpoints, and new pods matching the Service’s selector will automatically get added to the endpoints. Check the endpoints, and note that the IPs are the same as the pods created in the first step: +As mentioned previously, a Service is backed by a group of pods. These pods are exposed through `endpoints`. The Service's selector will be evaluated continuously and the results will be POSTed to an Endpoints object also named `nginxsvc`. When a pod dies, it is automatically removed from the endpoints, and new pods matching the Service's selector will automatically get added to the endpoints. Check the endpoints, and note that the IPs are the same as the pods created in the first step: {% highlight console %} -{% raw %} + $ kubectl describe svc nginxsvc Name: nginxsvc Namespace: default @@ -139,10 +121,10 @@ No events. $ kubectl get ep NAME ENDPOINTS nginxsvc 10.245.0.14:80,10.245.0.15:80 -{% endraw %} + {% endhighlight %} -You should now be able to curl the nginx Service on `10.0.116.146:80` from any node in your cluster. Note that the Service IP is completely virtual, it never hits the wire, if you’re curious about how this works you can read more about the [service proxy](services.html#virtual-ips-and-service-proxies). +You should now be able to curl the nginx Service on `10.0.116.146:80` from any node in your cluster. Note that the Service IP is completely virtual, it never hits the wire, if you're curious about how this works you can read more about the [service proxy](services.html#virtual-ips-and-service-proxies). ## Accessing the Service @@ -153,17 +135,17 @@ Kubernetes supports 2 primary modes of finding a Service - environment variables When a Pod is run on a Node, the kubelet adds a set of environment variables for each active Service. This introduces an ordering problem. To see why, inspect the environment of your running nginx pods: {% highlight console %} -{% raw %} + $ kubectl exec my-nginx-6isf4 -- printenv | grep SERVICE KUBERNETES_SERVICE_HOST=10.0.0.1 KUBERNETES_SERVICE_PORT=443 -{% endraw %} + {% endhighlight %} -Note there’s no mention of your Service. This is because you created the replicas before the Service. Another disadvantage of doing this is that the scheduler might put both pods on the same machine, which will take your entire Service down if it dies. We can do this the right way by killing the 2 pods and waiting for the replication controller to recreate them. This time around the Service exists *before* the replicas. This will given you scheduler level Service spreading of your pods (provided all your nodes have equal capacity), as well as the right environment variables: +Note there's no mention of your Service. This is because you created the replicas before the Service. Another disadvantage of doing this is that the scheduler might put both pods on the same machine, which will take your entire Service down if it dies. We can do this the right way by killing the 2 pods and waiting for the replication controller to recreate them. This time around the Service exists *before* the replicas. This will given you scheduler level Service spreading of your pods (provided all your nodes have equal capacity), as well as the right environment variables: {% highlight console %} -{% raw %} + $ kubectl scale rc my-nginx --replicas=0; kubectl scale rc my-nginx --replicas=2; $ kubectl get pods -l app=nginx -o wide NAME READY STATUS RESTARTS AGE NODE @@ -175,25 +157,25 @@ KUBERNETES_SERVICE_PORT=443 NGINXSVC_SERVICE_HOST=10.0.116.146 KUBERNETES_SERVICE_HOST=10.0.0.1 NGINXSVC_SERVICE_PORT=80 -{% endraw %} + {% endhighlight %} ### DNS -Kubernetes offers a DNS cluster addon Service that uses skydns to automatically assign dns names to other Services. You can check if it’s running on your cluster: +Kubernetes offers a DNS cluster addon Service that uses skydns to automatically assign dns names to other Services. You can check if it's running on your cluster: {% highlight console %} -{% raw %} + $ kubectl get services kube-dns --namespace=kube-system NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE kube-dns 10.179.240.10 53/UDP,53/TCP k8s-app=kube-dns 8d -{% endraw %} + {% endhighlight %} -If it isn’t running, you can [enable it](http://releases.k8s.io/release-1.1/cluster/addons/dns/README.md#how-do-i-configure-it). The rest of this section will assume you have a Service with a long lived IP (nginxsvc), and a dns server that has assigned a name to that IP (the kube-dns cluster addon), so you can talk to the Service from any pod in your cluster using standard methods (e.g. gethostbyname). Let’s create another pod to test this: +If it isn't running, you can [enable it](http://releases.k8s.io/release-1.1/cluster/addons/dns/README.md#how-do-i-configure-it). The rest of this section will assume you have a Service with a long lived IP (nginxsvc), and a dns server that has assigned a name to that IP (the kube-dns cluster addon), so you can talk to the Service from any pod in your cluster using standard methods (e.g. gethostbyname). Let's create another pod to test this: {% highlight yaml %} -{% raw %} + $ cat curlpod.yaml apiVersion: v1 kind: Pod @@ -208,13 +190,13 @@ spec: imagePullPolicy: IfNotPresent name: curlcontainer restartPolicy: Always -{% endraw %} + {% endhighlight %} And perform a lookup of the nginx Service {% highlight console %} -{% raw %} + $ kubectl create -f ./curlpod.yaml default/curlpod $ kubectl get pods curlpod @@ -226,7 +208,7 @@ Server: 10.0.0.10 Address 1: 10.0.0.10 Name: nginxsvc Address 1: 10.0.116.146 -{% endraw %} + {% endhighlight %} ## Securing the Service @@ -234,12 +216,12 @@ Address 1: 10.0.116.146 Till now we have only accessed the nginx server from within the cluster. Before exposing the Service to the internet, you want to make sure the communication channel is secure. For this, you will need: * Self signed certificates for https (unless you already have an identity certificate) * An nginx server configured to use the certificates -* A [secret](secrets.html) that makes the certificates accessible to pods +* A [secret](secrets) that makes the certificates accessible to pods -You can acquire all these from the [nginx https example](../../examples/https-nginx/README.html), in short: +You can acquire all these from the [nginx https example](../../examples/https-nginx/README), in short: {% highlight console %} -{% raw %} + $ make keys secret KEY=/tmp/nginx.key CERT=/tmp/nginx.crt SECRET=/tmp/secret.json $ kubectl create -f /tmp/secret.json secrets/nginxsecret @@ -247,13 +229,13 @@ $ kubectl get secrets NAME TYPE DATA default-token-il9rc kubernetes.io/service-account-token 1 nginxsecret Opaque 2 -{% endraw %} + {% endhighlight %} Now modify your nginx replicas to start a https server using the certificate in the secret, and the Service, to expose both ports (80 and 443): {% highlight yaml %} -{% raw %} + $ cat nginx-app.yaml apiVersion: v1 kind: Service @@ -298,7 +280,7 @@ spec: volumeMounts: - mountPath: /etc/nginx/ssl name: secret-volume -{% endraw %} + {% endhighlight %} Noteworthy points about the nginx-app manifest: @@ -307,25 +289,25 @@ Noteworthy points about the nginx-app manifest: - Each container has access to the keys through a volume mounted at /etc/nginx/ssl. This is setup *before* the nginx server is started. {% highlight console %} -{% raw %} + $ kubectl delete rc,svc -l app=nginx; kubectl create -f ./nginx-app.yaml replicationcontrollers/my-nginx services/nginxsvc services/nginxsvc replicationcontrollers/my-nginx -{% endraw %} + {% endhighlight %} At this point you can reach the nginx server from any node. {% highlight console %} -{% raw %} + $ kubectl get pods -o json | grep -i podip "podIP": "10.1.0.80", node $ curl -k https://10.1.0.80 ...

    Welcome to nginx!

    -{% endraw %} + {% endhighlight %} Note how we supplied the `-k` parameter to curl in the last step, this is because we don't know anything about the pods running nginx at certificate generation time, @@ -333,7 +315,7 @@ so we have to tell curl to ignore the CName mismatch. By creating a Service we l Lets test this from a pod (the same secret is being reused for simplicity, the pod only needs nginx.crt to access the Service): {% highlight console %} -{% raw %} + $ cat curlpod.yaml vapiVersion: v1 kind: ReplicationController @@ -371,7 +353,7 @@ $ kubectl exec curlpod -- curl https://nginxsvc --cacert /etc/nginx/ssl/nginx.cr ... Welcome to nginx! ... -{% endraw %} + {% endhighlight %} ## Exposing the Service @@ -379,7 +361,7 @@ $ kubectl exec curlpod -- curl https://nginxsvc --cacert /etc/nginx/ssl/nginx.cr For some parts of your applications you may want to expose a Service onto an external IP address. Kubernetes supports two ways of doing this: NodePorts and LoadBalancers. The Service created in the last section already used `NodePort`, so your nginx https replica is ready to serve traffic on the internet if your node has a public IP. {% highlight console %} -{% raw %} + $ kubectl get svc nginxsvc -o json | grep -i nodeport -C 5 { "name": "http", @@ -410,13 +392,13 @@ $ kubectl get nodes -o json | grep ExternalIP -C 2 $ curl https://104.197.63.17:30645 -k ...

    Welcome to nginx!

    -{% endraw %} + {% endhighlight %} Lets now recreate the Service to use a cloud load balancer, just change the `Type` of Service in the nginx-app.yaml from `NodePort` to `LoadBalancer`: {% highlight console %} -{% raw %} + $ kubectl delete rc, svc -l app=nginx $ kubectl create -f ./nginx-app.yaml $ kubectl get svc nginxsvc @@ -426,7 +408,7 @@ nginxsvc 10.179.252.126 162.222.184.144 80/TCP,81/TCP,82/TCP run=nginx2 $ curl https://162.22.184.144 -k ... Welcome to nginx! -{% endraw %} + {% endhighlight %} The IP address in the `EXTERNAL_IP` column is the one that is available on the public internet. The `CLUSTER_IP` is only available inside your @@ -434,7 +416,7 @@ cluster/private cloud network. ## What's next? -[Learn about more Kubernetes features that will help you run containers reliably in production.](production-pods.html) +[Learn about more Kubernetes features that will help you run containers reliably in production.](production-pods) diff --git a/v1.1/docs/user-guide/connecting-to-applications-port-forward.md b/v1.1/docs/user-guide/connecting-to-applications-port-forward.md index 7d7ee6ca7b..4903c457e1 100644 --- a/v1.1/docs/user-guide/connecting-to-applications-port-forward.md +++ b/v1.1/docs/user-guide/connecting-to-applications-port-forward.md @@ -1,59 +1,48 @@ ---- -title: "Connecting to applications: kubectl port-forward" -section: support ---- -kubectl port-forward forwards connections to a local port to a port on a pod. Its man page is available [here](kubectl/kubectl_port-forward.html). Compared to [kubectl proxy](accessing-the-cluster.html#using-kubectl-proxy), `kubectl port-forward` is more generic as it can forward TCP traffic while `kubectl proxy` can only forward HTTP traffic. This guide demonstrates how to use `kubectl port-forward` to connect to a Redis database, which may be useful for database debugging. - -## Creating a Redis master - -{% highlight console %} -{% raw %} -$ kubectl create examples/redis/redis-master.yaml -pods/redis-master -{% endraw %} -{% endhighlight %} - -wait until the Redis master pod is Running and Ready, - -{% highlight console %} -{% raw %} -$ kubectl get pods -NAME READY STATUS RESTARTS AGE -redis-master 2/2 Running 0 41s -{% endraw %} -{% endhighlight %} - - -## Connecting to the Redis master[a] - -The Redis master is listening on port 6397, to verify this, - -{% highlight console %} -{% raw %} -$ kubectl get pods redis-master -t='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}' -6379 -{% endraw %} -{% endhighlight %} - - -then we forward the port 6379 on the local workstation to the port 6379 of pod redis-master, - -{% highlight console %} -{% raw %} -$ kubectl port-forward redis-master 6379:6379 -I0710 14:43:38.274550 3655 portforward.go:225] Forwarding from 127.0.0.1:6379 -> 6379 -I0710 14:43:38.274797 3655 portforward.go:225] Forwarding from [::1]:6379 -> 6379 -{% endraw %} -{% endhighlight %} - -To verify the connection is successful, we run a redis-cli on the local workstation, - -{% highlight console %} -{% raw %} -$ redis-cli -127.0.0.1:6379> ping -PONG -{% endraw %} -{% endhighlight %} - +--- +title: "Connecting to applications: kubectl port-forward" +--- +kubectl port-forward forwards connections to a local port to a port on a pod. Its man page is available [here](kubectl/kubectl_port-forward). Compared to [kubectl proxy](accessing-the-cluster.html#using-kubectl-proxy), `kubectl port-forward` is more generic as it can forward TCP traffic while `kubectl proxy` can only forward HTTP traffic. This guide demonstrates how to use `kubectl port-forward` to connect to a Redis database, which may be useful for database debugging. + +## Creating a Redis master + +{% highlight console %} +$ kubectl create examples/redis/redis-master.yaml +pods/redis-master +{% endhighlight %} + +wait until the Redis master pod is Running and Ready, + +{% highlight console %} +$ kubectl get pods +NAME READY STATUS RESTARTS AGE +redis-master 2/2 Running 0 41s +{% endhighlight %} + + +## Connecting to the Redis master[a] + +The Redis master is listening on port 6397, to verify this, + +{% highlight console %} +$ kubectl get pods redis-master -t='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}' +6379 +{% endhighlight %} + + +then we forward the port 6379 on the local workstation to the port 6379 of pod redis-master, + +{% highlight console %} +$ kubectl port-forward redis-master 6379:6379 +I0710 14:43:38.274550 3655 portforward.go:225] Forwarding from 127.0.0.1:6379 -> 6379 +I0710 14:43:38.274797 3655 portforward.go:225] Forwarding from [::1]:6379 -> 6379 +{% endhighlight %} + +To verify the connection is successful, we run a redis-cli on the local workstation, + +{% highlight console %} +$ redis-cli +127.0.0.1:6379> ping +PONG +{% endhighlight %} + Now one can debug the database from the local workstation. \ No newline at end of file diff --git a/v1.1/docs/user-guide/connecting-to-applications-proxy.md b/v1.1/docs/user-guide/connecting-to-applications-proxy.md index 3fb7450a6a..eae9b8bf2e 100644 --- a/v1.1/docs/user-guide/connecting-to-applications-proxy.md +++ b/v1.1/docs/user-guide/connecting-to-applications-proxy.md @@ -1,33 +1,28 @@ ---- -title: "Connecting to applications: kubectl proxy and apiserver proxy" -section: support ---- -You have seen the [basics](accessing-the-cluster.html) about `kubectl proxy` and `apiserver proxy`. This guide shows how to use them together to access a service([kube-ui](ui.html)) running on the Kubernetes cluster from your workstation. - - -## Getting the apiserver proxy URL of kube-ui - -kube-ui is deployed as a cluster add-on. To find its apiserver proxy URL, - -{% highlight console %} -{% raw %} -$ kubectl cluster-info | grep "KubeUI" -KubeUI is running at https://173.255.119.104/api/v1/proxy/namespaces/kube-system/services/kube-ui -{% endraw %} -{% endhighlight %} - -if this command does not find the URL, try the steps [here](ui.html#accessing-the-ui). - - -## Connecting to the kube-ui service from your local workstation - -The above proxy URL is an access to the kube-ui service provided by the apiserver. To access it, you still need to authenticate to the apiserver. `kubectl proxy` can handle the authentication. - -{% highlight console %} -{% raw %} -$ kubectl proxy --port=8001 -Starting to serve on localhost:8001 -{% endraw %} -{% endhighlight %} - +--- +title: "Connecting to applications: kubectl proxy and apiserver proxy" +--- +You have seen the [basics](accessing-the-cluster) about `kubectl proxy` and `apiserver proxy`. This guide shows how to use them together to access a service([kube-ui](ui)) running on the Kubernetes cluster from your workstation. + + +## Getting the apiserver proxy URL of kube-ui + +kube-ui is deployed as a cluster add-on. To find its apiserver proxy URL, + +{% highlight console %} +$ kubectl cluster-info | grep "KubeUI" +KubeUI is running at https://173.255.119.104/api/v1/proxy/namespaces/kube-system/services/kube-ui +{% endhighlight %} + +if this command does not find the URL, try the steps [here](ui.html#accessing-the-ui). + + +## Connecting to the kube-ui service from your local workstation + +The above proxy URL is an access to the kube-ui service provided by the apiserver. To access it, you still need to authenticate to the apiserver. `kubectl proxy` can handle the authentication. + +{% highlight console %} +$ kubectl proxy --port=8001 +Starting to serve on localhost:8001 +{% endhighlight %} + Now you can access the kube-ui service on your local workstation at [http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kube-ui](http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kube-ui) \ No newline at end of file diff --git a/v1.1/docs/user-guide/container-environment.md b/v1.1/docs/user-guide/container-environment.md index 0b3583af70..d4ef33c6e3 100644 --- a/v1.1/docs/user-guide/container-environment.md +++ b/v1.1/docs/user-guide/container-environment.md @@ -1,26 +1,7 @@ --- title: "Kubernetes Container Environment" --- - - -# Kubernetes Container Environment - -**Table of Contents** - - -- [Kubernetes Container Environment](#kubernetes-container-environment) - - [Overview](#overview) - - [Cluster Information](#cluster-information) - - [Container Information](#container-information) - - [Cluster Information](#cluster-information) - - [Container Hooks](#container-hooks) - - [Hook Details](#hook-details) - - [Hook Handler Execution](#hook-handler-execution) - - [Hook delivery guarantees](#hook-delivery-guarantees) - - [Hook Handler Implementations](#hook-handler-implementations) - - - +{% include pagetoc.html %} ## Overview @@ -29,7 +10,7 @@ This document describes the environment for Kubelet managed containers on a Kube This cluster information makes it possible to build applications that are *cluster aware*. Additionally, the Kubernetes container environment defines a series of hooks that are surfaced to optional hook handlers defined as part of individual containers.  Container hooks are somewhat analogous to operating system signals in a traditional process model.   However these hooks are designed to make it easier to build reliable, scalable cloud applications in the Kubernetes cluster.  Containers that participate in this cluster lifecycle become *cluster native*. -Another important part of the container environment is the file system that is available to the container. In Kubernetes, the filesystem is a combination of an [image](images.html) and one or more [volumes](volumes.html). +Another important part of the container environment is the file system that is available to the container. In Kubernetes, the filesystem is a combination of an [image](images) and one or more [volumes](volumes). The following sections describe both the cluster information provided to containers, as well as the hooks and life-cycle that allows containers to interact with the management system. @@ -42,7 +23,7 @@ There are two types of information that are available within the container envir Currently, the Pod name for the pod in which the container is running is set as the hostname of the container, and is accessible through all calls to access the hostname within the container (e.g. the hostname command, or the [gethostname][1] function call in libc), but this is planned to change in the future and should not be used. -The Pod name and namespace are also available as environment variables via the [downward API](downward-api.html). Additionally, user-defined environment variables from the pod definition, are also available to the container, as are any environment variables specified statically in the Docker image. +The Pod name and namespace are also available as environment variables via the [downward API](downward-api). Additionally, user-defined environment variables from the pod definition, are also available to the container, as are any environment variables specified statically in the Docker image. In the future, we anticipate expanding this information with richer information about the container.  Examples include available memory, number of restarts, and in general any state that you could get from the call to GET /pods on the API server. @@ -53,10 +34,10 @@ Currently the list of all services that are running at the time when the contain For a service named **foo** that maps to a container port named **bar**, the following variables are defined: {% highlight sh %} -{% raw %} + FOO_SERVICE_HOST= FOO_SERVICE_PORT= -{% endraw %} + {% endhighlight %} Services have dedicated IP address, and are also surfaced to the container via DNS (If [DNS addon](http://releases.k8s.io/release-1.1/cluster/addons/dns/) is enabled).  Of course DNS is still not an enumerable protocol, so we will continue to provide environment variables so that containers can do discovery. diff --git a/v1.1/docs/user-guide/containers.md b/v1.1/docs/user-guide/containers.md index 2bbfe871c2..cc6f2d0dd8 100644 --- a/v1.1/docs/user-guide/containers.md +++ b/v1.1/docs/user-guide/containers.md @@ -1,10 +1,6 @@ --- title: "Containers with Kubernetes" --- - - -# Containers with Kubernetes - ## Containers and commands So far the Pods we've seen have all used the `image` field to indicate what process Kubernetes @@ -51,7 +47,7 @@ Here are examples for these rules in table format By default, Docker containers are "unprivileged" and cannot, for example, run a Docker daemon inside a Docker container. We can have fine grain control over the capabilities using cap-add and cap-drop.More details [here](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration). -The relationship between Docker's capabilities and [Linux capabilities](http://man7.org/linux/man-pages/man7/capabilities.7.html) +The relationship between Docker's capabilities and [Linux capabilities](http://man7.org/linux/man-pages/man7/capabilities.7) | Docker's capabilities | Linux capabilities | | ---- | ---- | diff --git a/v1.1/docs/user-guide/debugging-services.md b/v1.1/docs/user-guide/debugging-services.md index 8e615617dc..5ad5821214 100644 --- a/v1.1/docs/user-guide/debugging-services.md +++ b/v1.1/docs/user-guide/debugging-services.md @@ -1,37 +1,13 @@ --- title: "My Service is not working - how to debug" --- - - -# My Service is not working - how to debug - An issue that comes up rather frequently for new installations of Kubernetes is that `Services` are not working properly. You've run all your `Pod`s and `ReplicationController`s, but you get no response when you try to access them. This document will hopefully help you to figure out what's going wrong. -**Table of Contents** - +{% include pagetoc.html %} -- [My Service is not working - how to debug](#my-service-is-not-working---how-to-debug) - - [Conventions](#conventions) - - [Running commands in a Pod](#running-commands-in-a-pod) - - [Setup](#setup) - - [Does the Service exist?](#does-the-service-exist) - - [Does the Service work by DNS?](#does-the-service-work-by-dns) - - [Does any Service exist in DNS?](#does-any-service-exist-in-dns) - - [Does the Service work by IP?](#does-the-service-work-by-ip) - - [Is the Service correct?](#is-the-service-correct) - - [Does the Service have any Endpoints?](#does-the-service-have-any-endpoints) - - [Are the Pods working?](#are-the-pods-working) - - [Is the kube-proxy working?](#is-the-kube-proxy-working) - - [Is kube-proxy running?](#is-kube-proxy-running) - - [Is kube-proxy writing iptables rules?](#is-kube-proxy-writing-iptables-rules) - - [Is kube-proxy proxying?](#is-kube-proxy-proxying) - - [Seek help](#seek-help) - - [More information](#more-information) - - ## Conventions @@ -43,28 +19,28 @@ clear what is expected, this document will use the following conventions. If the command "COMMAND" is expected to run in a `Pod` and produce "OUTPUT": {% highlight console %} -{% raw %} + u@pod$ COMMAND OUTPUT -{% endraw %} + {% endhighlight %} If the command "COMMAND" is expected to run on a `Node` and produce "OUTPUT": {% highlight console %} -{% raw %} + u@node$ COMMAND OUTPUT -{% endraw %} + {% endhighlight %} If the command is "kubectl ARGS": {% highlight console %} -{% raw %} + $ kubectl ARGS OUTPUT -{% endraw %} + {% endhighlight %} ## Running commands in a Pod @@ -74,7 +50,7 @@ sees. Kubernetes does not directly support interactive `Pod`s (yet), but you ca approximate it: {% highlight console %} -{% raw %} + $ cat < -{% endraw %} + {% endhighlight %} or {% highlight console %} -{% raw %} + $ kubectl exec -ti busybox-sleep sh / # -{% endraw %} + {% endhighlight %} ## Setup @@ -117,21 +93,21 @@ probably debugging your own `Service` you can substitute your own details, or yo can follow along and get a second data point. {% highlight console %} -{% raw %} + $ kubectl run hostnames --image=gcr.io/google_containers/serve_hostname \ --labels=app=hostnames \ --port=9376 \ --replicas=3 CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS hostnames hostnames gcr.io/google_containers/serve_hostname app=hostnames 3 -{% endraw %} + {% endhighlight %} Note that this is the same as if you had started the `ReplicationController` with the following YAML: {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: ReplicationController metadata: @@ -151,19 +127,19 @@ spec: ports: - containerPort: 9376 protocol: TCP -{% endraw %} + {% endhighlight %} Confirm your `Pod`s are running: {% highlight console %} -{% raw %} + $ kubectl get pods -l app=hostnames NAME READY STATUS RESTARTS AGE hostnames-0uton 1/1 Running 0 12s hostnames-bvc05 1/1 Running 0 12s hostnames-yp2kp 1/1 Running 0 12s -{% endraw %} + {% endhighlight %} ## Does the Service exist? @@ -177,53 +153,53 @@ have another `Pod` that consumes this `Service` by name you would get something like: {% highlight console %} -{% raw %} + u@pod$ wget -qO- hostnames wget: bad address 'hostname' -{% endraw %} + {% endhighlight %} or: {% highlight console %} -{% raw %} + u@pod$ echo $HOSTNAMES_SERVICE_HOST -{% endraw %} + {% endhighlight %} So the first thing to check is whether that `Service` actually exists: {% highlight console %} -{% raw %} + $ kubectl get svc hostnames Error from server: service "hostnames" not found -{% endraw %} + {% endhighlight %} So we have a culprit, let's create the `Service`. As before, this is for the walk-through - you can use your own `Service`'s details here. {% highlight console %} -{% raw %} + $ kubectl expose rc hostnames --port=80 --target-port=9376 service "hostnames" exposed -{% endraw %} + {% endhighlight %} And read it back, just to be sure: {% highlight console %} -{% raw %} + $ kubectl get svc hostnames NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE hostnames 10.0.0.1 80/TCP run=hostnames 1h -{% endraw %} + {% endhighlight %} As before, this is the same as if you had started the `Service` with YAML: {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Service metadata: @@ -236,7 +212,7 @@ spec: protocol: TCP port: 80 targetPort: 9376 -{% endraw %} + {% endhighlight %} Now you can confirm that the `Service` exists. @@ -246,42 +222,42 @@ Now you can confirm that the `Service` exists. From a `Pod` in the same `Namespace`: {% highlight console %} -{% raw %} + u@pod$ nslookup hostnames Server: 10.0.0.10 Address: 10.0.0.10#53 Name: hostnames Address: 10.0.1.175 -{% endraw %} + {% endhighlight %} If this fails, perhaps your `Pod` and `Service` are in different `Namespace`s, try a namespace-qualified name: {% highlight console %} -{% raw %} + u@pod$ nslookup hostnames.default Server: 10.0.0.10 Address: 10.0.0.10#53 Name: hostnames.default Address: 10.0.1.175 -{% endraw %} + {% endhighlight %} If this works, you'll need to ensure that `Pod`s and `Service`s run in the same `Namespace`. If this still fails, try a fully-qualified name: {% highlight console %} -{% raw %} + u@pod$ nslookup hostnames.default.svc.cluster.local Server: 10.0.0.10 Address: 10.0.0.10#53 Name: hostnames.default.svc.cluster.local Address: 10.0.1.175 -{% endraw %} + {% endhighlight %} Note the suffix here: "default.svc.cluster.local". The "default" is the @@ -292,14 +268,14 @@ You can also try this from a `Node` in the cluster (note: 10.0.0.10 is my DNS `Service`): {% highlight console %} -{% raw %} + u@node$ nslookup hostnames.default.svc.cluster.local 10.0.0.10 Server: 10.0.0.10 Address: 10.0.0.10#53 Name: hostnames.default.svc.cluster.local Address: 10.0.1.175 -{% endraw %} + {% endhighlight %} If you are able to do a fully-qualified name lookup but not a relative one, you @@ -316,14 +292,14 @@ can take a step back and see what else is not working. The Kubernetes master `Service` should always work: {% highlight console %} -{% raw %} + u@pod$ nslookup kubernetes.default Server: 10.0.0.10 Address 1: 10.0.0.10 Name: kubernetes Address 1: 10.0.0.1 -{% endraw %} + {% endhighlight %} If this fails, you might need to go to the kube-proxy section of this doc, or @@ -336,7 +312,7 @@ The next thing to test is whether your `Service` works at all. From a `Node` in your cluster, access the `Service`'s IP (from `kubectl get` above). {% highlight console %} -{% raw %} + u@node$ curl 10.0.1.175:80 hostnames-0uton @@ -345,7 +321,7 @@ hostnames-yp2kp u@node$ curl 10.0.1.175:80 hostnames-bvc05 -{% endraw %} + {% endhighlight %} If your `Service` is working, you should get correct responses. If not, there @@ -358,7 +334,7 @@ It might sound silly, but you should really double and triple check that your verify it: {% highlight console %} -{% raw %} + $ kubectl get service hostnames -o json { "kind": "Service", @@ -395,7 +371,7 @@ $ kubectl get service hostnames -o json "loadBalancer": {} } } -{% endraw %} + {% endhighlight %} Is the port you are trying to access in `spec.ports[]`? Is the `targetPort` @@ -413,13 +389,13 @@ actually being selected by the `Service`. Earlier we saw that the `Pod`s were running. We can re-check that: {% highlight console %} -{% raw %} + $ kubectl get pods -l app=hostnames NAME READY STATUS RESTARTS AGE hostnames-0uton 1/1 Running 0 1h hostnames-bvc05 1/1 Running 0 1h hostnames-yp2kp 1/1 Running 0 1h -{% endraw %} + {% endhighlight %} The "AGE" column says that these `Pod`s are about an hour old, which implies that @@ -430,11 +406,11 @@ has. Inside the Kubernetes system is a control loop which evaluates the selector of every `Service` and save the results into an `Endpoints` object. {% highlight console %} -{% raw %} + $ kubectl get endpoints hostnames NAME ENDPOINTS hostnames 10.244.0.5:9376,10.244.0.6:9376,10.244.0.7:9376 -{% endraw %} + {% endhighlight %} This confirms that the control loop has found the correct `Pod`s for your @@ -449,7 +425,7 @@ Let's check that the `Pod`s are actually working - we can bypass the `Service` mechanism and go straight to the `Pod`s. {% highlight console %} -{% raw %} + u@pod$ wget -qO- 10.244.0.5:9376 hostnames-0uton @@ -458,7 +434,7 @@ hostnames-bvc05 u@pod$ wget -qO- 10.244.0.7:9376 hostnames-yp2kp -{% endraw %} + {% endhighlight %} We expect each `Pod` in the `Endpoints` list to return its own hostname. If @@ -479,10 +455,10 @@ Confirm that `kube-proxy` is running on your `Node`s. You should get something like the below: {% highlight console %} -{% raw %} + u@node$ ps auxw | grep kube-proxy root 4194 0.4 0.1 101864 17696 ? Sl Jul04 25:43 /usr/local/bin/kube-proxy --master=https://kubernetes-master --kubeconfig=/var/lib/kube-proxy/kubeconfig --v=2 -{% endraw %} + {% endhighlight %} Next, confirm that it is not failing something obvious, like contacting the @@ -492,7 +468,7 @@ depends on your `Node` OS. On some OSes it is a file, such as should see something like: {% highlight console %} -{% raw %} + I0707 17:34:53.945651 30031 server.go:88] Running in resource-only container "/kube-proxy" I0707 17:34:53.945921 30031 proxier.go:121] Setting proxy IP to 10.240.115.247 and initializing iptables I0707 17:34:54.053023 30031 roundrobin.go:262] LoadBalancerRR: Setting endpoints for default/kubernetes: to [10.240.169.188:443] @@ -511,7 +487,7 @@ I0707 17:34:54.902313 30031 proxysocket.go:130] Accepted TCP connection from 1 I0707 17:34:54.903107 30031 proxysocket.go:130] Accepted TCP connection from 10.244.3.3:42671 to 10.244.3.1:40074 I0707 17:35:46.015868 30031 proxysocket.go:246] New UDP connection from 10.244.3.2:57493 I0707 17:35:46.017061 30031 proxysocket.go:246] New UDP connection from 10.244.3.2:55471 -{% endraw %} + {% endhighlight %} If you see error messages about not being able to contact the master, you @@ -524,11 +500,11 @@ rules which implement `Service`s. Let's check that those rules are getting written. {% highlight console %} -{% raw %} + u@node$ iptables-save | grep hostnames -A KUBE-PORTALS-CONTAINER -d 10.0.1.175/32 -p tcp -m comment --comment "default/hostnames:default" -m tcp --dport 80 -j REDIRECT --to-ports 48577 -A KUBE-PORTALS-HOST -d 10.0.1.175/32 -p tcp -m comment --comment "default/hostnames:default" -m tcp --dport 80 -j DNAT --to-destination 10.240.115.247:48577 -{% endraw %} + {% endhighlight %} There should be 2 rules for each port on your `Service` (just one in this @@ -541,10 +517,10 @@ then look at the logs again. Assuming you do see the above rules, try again to access your `Service` by IP: {% highlight console %} -{% raw %} + u@node$ curl 10.0.1.175:80 hostnames-0uton -{% endraw %} + {% endhighlight %} If this fails, we can try accessing the proxy directly. Look back at the @@ -553,18 +529,18 @@ using for your `Service`. In the above examples it is "48577". Now connect to that: {% highlight console %} -{% raw %} + u@node$ curl localhost:48577 hostnames-yp2kp -{% endraw %} + {% endhighlight %} If this still fails, look at the `kube-proxy` logs for specific lines like: {% highlight console %} -{% raw %} + Setting endpoints for default/hostnames:default to [10.244.0.5:9376 10.244.0.6:9376 10.244.0.7:9376] -{% endraw %} + {% endhighlight %} If you don't see those, try restarting `kube-proxy` with the `-V` flag set to 4, and @@ -585,7 +561,7 @@ Contact us on ## More information -Visit [troubleshooting document](../troubleshooting.html) for more information. +Visit [troubleshooting document](../troubleshooting) for more information. diff --git a/v1.1/docs/user-guide/deploying-applications.md b/v1.1/docs/user-guide/deploying-applications.md index 3ee955d1a7..d2927dcd84 100644 --- a/v1.1/docs/user-guide/deploying-applications.md +++ b/v1.1/docs/user-guide/deploying-applications.md @@ -1,34 +1,20 @@ --- title: "Kubernetes User Guide: Managing Applications: Deploying continuously running applications" --- +{% include pagetoc.html %} - -# Kubernetes User Guide: Managing Applications: Deploying continuously running applications - -**Table of Contents** - - -- [Kubernetes User Guide: Managing Applications: Deploying continuously running applications](#kubernetes-user-guide-managing-applications-deploying-continuously-running-applications) - - [Launching a set of replicas using a configuration file](#launching-a-set-of-replicas-using-a-configuration-file) - - [Viewing replication controller status](#viewing-replication-controller-status) - - [Deleting replication controllers](#deleting-replication-controllers) - - [Labels](#labels) - - [What's next?](#whats-next) - - - -You previously read about how to quickly deploy a simple replicated application using [`kubectl run`](quick-start.html) and how to configure and launch single-run containers using pods ([Configuring containers](configuring-containers.html)). Here you’ll use the configuration-based approach to deploy a continuously running, replicated application. +You previously read about how to quickly deploy a simple replicated application using [`kubectl run`](quick-start) and how to configure and launch single-run containers using pods ([Configuring containers](configuring-containers)). Here you'll use the configuration-based approach to deploy a continuously running, replicated application. ## Launching a set of replicas using a configuration file -Kubernetes creates and manages sets of replicated containers (actually, replicated [Pods](pods.html)) using [*Replication Controllers*](replication-controller.html). +Kubernetes creates and manages sets of replicated containers (actually, replicated [Pods](pods)) using [*Replication Controllers*](replication-controller). -A replication controller simply ensures that a specified number of pod "replicas" are running at any one time. If there are too many, it will kill some. If there are too few, it will start more. It’s analogous to Google Compute Engine’s [Instance Group Manager](https://cloud.google.com/compute/docs/instance-groups/manager/) or AWS’s [Auto-scaling Group](http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/AutoScalingGroup.html) (with no scaling policies). +A replication controller simply ensures that a specified number of pod "replicas" are running at any one time. If there are too many, it will kill some. If there are too few, it will start more. It's analogous to Google Compute Engine's [Instance Group Manager](https://cloud.google.com/compute/docs/instance-groups/manager/) or AWS's [Auto-scaling Group](http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/AutoScalingGroup) (with no scaling policies). -The replication controller created to run nginx by `kubectl run` in the [Quick start](quick-start.html) could be specified using YAML as follows: +The replication controller created to run nginx by `kubectl run` in the [Quick start](quick-start) could be specified using YAML as follows: {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: ReplicationController metadata: @@ -45,10 +31,10 @@ spec: image: nginx ports: - containerPort: 80 -{% endraw %} + {% endhighlight %} -Some differences compared to specifying just a pod are that the `kind` is `ReplicationController`, the number of `replicas` desired is specified, and the pod specification is under the `template` field. The names of the pods don’t need to be specified explicitly because they are generated from the name of the replication controller. +Some differences compared to specifying just a pod are that the `kind` is `ReplicationController`, the number of `replicas` desired is specified, and the pod specification is under the `template` field. The names of the pods don't need to be specified explicitly because they are generated from the name of the replication controller. View the [replication controller API object](http://kubernetes.io/v1.1/docs/api-reference/v1/definitions.html#_v1_replicationcontroller) to view the list of supported fields. @@ -56,10 +42,10 @@ to view the list of supported fields. This replication controller can be created using `create`, just as with pods: {% highlight console %} -{% raw %} + $ kubectl create -f ./nginx-rc.yaml replicationcontrollers/my-nginx -{% endraw %} + {% endhighlight %} Unlike in the case where you directly create pods, a replication controller replaces pods that are deleted or terminated for any reason, such as in the case of node failure. For this reason, we recommend that you use a replication controller for a continuously running application even if your application requires only a single pod, in which case you can omit `replicas` and it will default to a single replica. @@ -69,11 +55,11 @@ Unlike in the case where you directly create pods, a replication controller repl You can view the replication controller you created using `get`: {% highlight console %} -{% raw %} + $ kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS my-nginx nginx nginx app=nginx 2 -{% endraw %} + {% endhighlight %} This tells you that your controller will ensure that you have two nginx replicas. @@ -81,23 +67,23 @@ This tells you that your controller will ensure that you have two nginx replicas You can see those replicas using `get`, just as with pods you created directly: {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY STATUS RESTARTS AGE my-nginx-065jq 1/1 Running 0 51s my-nginx-buaiq 1/1 Running 0 51s -{% endraw %} + {% endhighlight %} ## Deleting replication controllers -When you want to kill your application, delete your replication controller, as in the [Quick start](quick-start.html): +When you want to kill your application, delete your replication controller, as in the [Quick start](quick-start): {% highlight console %} -{% raw %} + $ kubectl delete rc my-nginx replicationcontrollers/my-nginx -{% endraw %} + {% endhighlight %} By default, this will also cause the pods managed by the replication controller to be deleted. If there were a large number of pods, this may take a while to complete. If you want to leave the pods running, specify `--cascade=false`. @@ -106,41 +92,41 @@ If you try to delete the pods before deleting the replication controller, it wil ## Labels -Kubernetes uses user-defined key-value attributes called [*labels*](labels.html) to categorize and identify sets of resources, such as pods and replication controllers. The example above specified a single label in the pod template, with key `app` and value `nginx`. All pods created carry that label, which can be viewed using `-L`: +Kubernetes uses user-defined key-value attributes called [*labels*](labels) to categorize and identify sets of resources, such as pods and replication controllers. The example above specified a single label in the pod template, with key `app` and value `nginx`. All pods created carry that label, which can be viewed using `-L`: {% highlight console %} -{% raw %} + $ kubectl get pods -L app NAME READY STATUS RESTARTS AGE APP my-nginx-afv12 0/1 Running 0 3s nginx my-nginx-lg99z 0/1 Running 0 3s nginx -{% endraw %} + {% endhighlight %} -The labels from the pod template are copied to the replication controller’s labels by default, as well -- all resources in Kubernetes support labels: +The labels from the pod template are copied to the replication controller's labels by default, as well -- all resources in Kubernetes support labels: {% highlight console %} -{% raw %} + $ kubectl get rc my-nginx -L app CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS APP my-nginx nginx nginx app=nginx 2 nginx -{% endraw %} + {% endhighlight %} -More importantly, the pod template’s labels are used to create a [`selector`](labels.html#label-selectors) that will match pods carrying those labels. You can see this field by requesting it using the [Go template output format of `kubectl get`](kubectl/kubectl_get.html): +More importantly, the pod template's labels are used to create a [`selector`](labels.html#label-selectors) that will match pods carrying those labels. You can see this field by requesting it using the [Go template output format of `kubectl get`](kubectl/kubectl_get): {% highlight console %} -{% raw %} + $ kubectl get rc my-nginx -o template --template="{{.spec.selector}}" map[app:nginx] -{% endraw %} + {% endhighlight %} -You could also specify the `selector` explicitly, such as if you wanted to specify labels in the pod template that you didn’t want to select on, but you should ensure that the selector will match the labels of the pods created from the pod template, and that it won’t match pods created by other replication controllers. The most straightforward way to ensure the latter is to create a unique label value for the replication controller, and to specify it in both the pod template’s labels and in the selector. +You could also specify the `selector` explicitly, such as if you wanted to specify labels in the pod template that you didn't want to select on, but you should ensure that the selector will match the labels of the pods created from the pod template, and that it won't match pods created by other replication controllers. The most straightforward way to ensure the latter is to create a unique label value for the replication controller, and to specify it in both the pod template's labels and in the selector. ## What's next? -[Learn about exposing applications to users and clients, and connecting tiers of your application together.](connecting-applications.html) +[Learn about exposing applications to users and clients, and connecting tiers of your application together.](connecting-applications) diff --git a/v1.1/docs/user-guide/deployments.md b/v1.1/docs/user-guide/deployments.md index d46f114971..79ac796dd8 100644 --- a/v1.1/docs/user-guide/deployments.md +++ b/v1.1/docs/user-guide/deployments.md @@ -1,34 +1,7 @@ --- title: "Deployments" --- - - -# Deployments - -**Table of Contents** - - -- [Deployments](#deployments) - - [What is a _Deployment_?](#what-is-a-deployment) - - [Enabling Deployments on kubernetes cluster](#enabling-deployments-on-kubernetes-cluster) - - [Creating a Deployment](#creating-a-deployment) - - [Updating a Deployment](#updating-a-deployment) - - [Multiple Updates](#multiple-updates) - - [Writing a Deployment Spec](#writing-a-deployment-spec) - - [Pod Template](#pod-template) - - [Replicas](#replicas) - - [Selector](#selector) - - [Unique Label Key](#unique-label-key) - - [Strategy](#strategy) - - [Recreate Deployment](#recreate-deployment) - - [Rolling Update Deployment](#rolling-update-deployment) - - [Max Unavailable](#max-unavailable) - - [Max Surge](#max-surge) - - [Min Ready Seconds](#min-ready-seconds) - - [Alternative to Deployments](#alternative-to-deployments) - - [kubectl rolling update](#kubectl-rolling-update) - - +{% include pagetoc.html %} ## What is a _Deployment_? @@ -64,7 +37,7 @@ bring up 3 nginx pods. {% highlight yaml %} -{% raw %} + apiVersion: extensions/v1beta1 kind: Deployment metadata: @@ -81,7 +54,7 @@ spec: image: nginx:1.7.9 ports: - containerPort: 80 -{% endraw %} + {% endhighlight %} [Download example](nginx-deployment.yaml) @@ -90,20 +63,20 @@ spec: Run the example by downloading the example file and then running this command: {% highlight console %} -{% raw %} + $ kubectl create -f docs/user-guide/nginx-deployment.yaml deployment "nginx-deployment" created -{% endraw %} + {% endhighlight %} Running a get immediately will give: {% highlight console %} -{% raw %} + $ kubectl get deployments NAME UPDATEDREPLICAS AGE nginx-deployment 0/3 8s -{% endraw %} + {% endhighlight %} This indicates that deployment is trying to update 3 replicas. It has not @@ -112,33 +85,33 @@ updated any one of those yet. Running a get again after a minute, will give: {% highlight console %} -{% raw %} + $ kubectl get deployments NAME UPDATEDREPLICAS AGE nginx-deployment 3/3 1m -{% endraw %} + {% endhighlight %} This indicates that deployent has created all the 3 replicas. Running ```kubectl get rc``` and ```kubectl get pods``` will show the replication controller (RC) and pods created. {% highlight console %} -{% raw %} + $ kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE REPLICAS AGE deploymentrc-1975012602 nginx nginx:1.7.9 deployment.kubernetes.io/podTemplateHash=1975012602,app=nginx 3 2m -{% endraw %} + {% endhighlight %} {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY STATUS RESTARTS AGE deploymentrc-1975012602-4f2tb 1/1 Running 0 1m deploymentrc-1975012602-j975u 1/1 Running 0 1m deploymentrc-1975012602-uashb 1/1 Running 0 1m -{% endraw %} + {% endhighlight %} The created RC will ensure that there are 3 nginx pods at all time. @@ -152,7 +125,7 @@ For this, we update our deployment to be as follows: {% highlight yaml %} -{% raw %} + apiVersion: extensions/v1beta1 kind: Deployment metadata: @@ -169,7 +142,7 @@ spec: image: nginx:1.9.1 ports: - containerPort: 80 -{% endraw %} + {% endhighlight %} [Download example](new-nginx-deployment.yaml) @@ -177,20 +150,20 @@ spec: {% highlight console %} -{% raw %} + $ kubectl apply -f docs/user-guide/new-nginx-deployment.yaml deployment "nginx-deployment" configured -{% endraw %} + {% endhighlight %} Running a get immediately will still give: {% highlight console %} -{% raw %} + $ kubectl get deployments NAME UPDATEDREPLICAS AGE nginx-deployment 3/3 8s -{% endraw %} + {% endhighlight %} This indicates that deployment status has not been updated yet (it is still @@ -198,11 +171,11 @@ showing old status). Running a get again after a minute, will give: {% highlight console %} -{% raw %} + $ kubectl get deployments NAME UPDATEDREPLICAS AGE nginx-deployment 1/3 1m -{% endraw %} + {% endhighlight %} This indicates that deployment has updated one of the three pods that it needs @@ -210,35 +183,35 @@ to update. Eventually, it will get around to updating all the pods. {% highlight console %} -{% raw %} + $ kubectl get deployments NAME UPDATEDREPLICAS AGE nginx-deployment 3/3 3m -{% endraw %} + {% endhighlight %} We can run ```kubectl get rc``` to see that deployment updated the pods by creating a new RC which it scaled up to 3 and scaled down the old RC to 0. {% highlight console %} -{% raw %} + kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE deploymentrc-1562004724 nginx nginx:1.9.1 deployment.kubernetes.io/podTemplateHash=1562004724,app=nginx 3 5m deploymentrc-1975012602 nginx nginx:1.7.9 deployment.kubernetes.io/podTemplateHash=1975012602,app=nginx 0 7m -{% endraw %} + {% endhighlight %} Running get pods, will only show the new pods. {% highlight console %} -{% raw %} + kubectl get pods NAME READY STATUS RESTARTS AGE deploymentrc-1562004724-0tgk5 1/1 Running 0 9m deploymentrc-1562004724-1rkfl 1/1 Running 0 8m deploymentrc-1562004724-6v702 1/1 Running 0 8m -{% endraw %} + {% endhighlight %} Next time we want to update pods, we can just update the deployment again. @@ -250,7 +223,7 @@ it first created a new pod, then deleted some old pods and created new ones. It does not kill old pods until a sufficient number of new pods have come up. {% highlight console %} -{% raw %} + $ kubectl describe deployments Name: nginx-deployment Namespace: default @@ -264,13 +237,13 @@ OldReplicationControllers: deploymentrc-1562004724 (3/3 replicas created) NewReplicationController: Events: FirstSeen LastSeen Count From SubobjectPath Reason Message - ───────── ──────── ───── ──── ───────────── ────── ─────── + '��'��'��'��'��'��'��'��'�� '��'��'��'��'��'��'��'�� '��'��'��'��'�� '��'��'��'�� '��'��'��'��'��'��'��'��'��'��'��'��'�� '��'��'��'��'��'�� '��'��'��'��'��'��'�� 10m 10m 1 {deployment-controller } ScalingRC Scaled up rc deploymentrc-1975012602 to 3 2m 2m 1 {deployment-controller } ScalingRC Scaled up rc deploymentrc-1562004724 to 1 2m 2m 1 {deployment-controller } ScalingRC Scaled down rc deploymentrc-1975012602 to 1 1m 1m 1 {deployment-controller } ScalingRC Scaled up rc deploymentrc-1562004724 to 3 1m 1m 1 {deployment-controller } ScalingRC Scaled down rc deploymentrc-1975012602 to 0 -{% endraw %} + {% endhighlight %} Here we see that when we first created the deployment, it created an RC and scaled it up to 3 replicas directly. @@ -300,7 +273,7 @@ before changing course. As with all other Kubernetes configs, a Deployment needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, -see [here](deploying-applications.html), [here](configuring-containers.html), and [here](working-with-resources.html). +see [here](deploying-applications), [here](configuring-containers), and [here](working-with-resources). A Deployment also needs a [`.spec` section](../devel/api-conventions.html#spec-and-status). @@ -309,7 +282,7 @@ A Deployment also needs a [`.spec` section](../devel/api-conventions.html#spec-a The `.spec.template` is the only required field of the `.spec`. The `.spec.template` is a [pod template](replication-controller.html#pod-template). It has exactly -the same schema as a [pod](pods.html), except it is nested and does not have an +the same schema as a [pod](pods), except it is nested and does not have an `apiVersion` or `kind`. ### Replicas @@ -397,8 +370,5 @@ Note: This is not implemented yet. ### kubectl rolling update -[Kubectl rolling update](kubectl/kubectl_rolling-update.html) also updates pods and replication controllers in a similar fashion. +[Kubectl rolling update](kubectl/kubectl_rolling-update) also updates pods and replication controllers in a similar fashion. But deployments is declarative and is server side. - - - diff --git a/v1.1/docs/user-guide/docker-cli-to-kubectl.md b/v1.1/docs/user-guide/docker-cli-to-kubectl.md index 50a4eb01b3..8fe87f153b 100644 --- a/v1.1/docs/user-guide/docker-cli-to-kubectl.md +++ b/v1.1/docs/user-guide/docker-cli-to-kubectl.md @@ -1,64 +1,46 @@ --- title: "kubectl for docker users" --- - - -# kubectl for docker users - In this doc, we introduce the Kubernetes command line for interacting with the api to docker-cli users. The tool, kubectl, is designed to be familiar to docker-cli users but there are a few necessary differences. Each section of this doc highlights a docker subcommand explains the kubectl equivalent. -**Table of Contents** - - -- [kubectl for docker users](#kubectl-for-docker-users) - - [docker run](#docker-run) - - [docker ps](#docker-ps) - - [docker attach](#docker-attach) - - [docker exec](#docker-exec) - - [docker logs](#docker-logs) - - [docker stop and docker rm](#docker-stop-and-docker-rm) - - [docker login](#docker-login) - - [docker version](#docker-version) - - [docker info](#docker-info) - - +{% include pagetoc.html %} #### docker run -How do I run an nginx container and expose it to the world? Checkout [kubectl run](kubectl/kubectl_run.html). +How do I run an nginx container and expose it to the world? Checkout [kubectl run](kubectl/kubectl_run). With docker: {% highlight console %} -{% raw %} + $ docker run -d --restart=always -e DOMAIN=cluster --name nginx-app -p 80:80 nginx a9ec34d9878748d2f33dc20cb25c714ff21da8d40558b45bfaec9955859075d0 $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES a9ec34d98787 nginx "nginx -g 'daemon of 2 seconds ago Up 2 seconds 0.0.0.0:80->80/tcp, 443/tcp nginx-app -{% endraw %} + {% endhighlight %} With kubectl: {% highlight console %} -{% raw %} + # start the pod running nginx $ kubectl run --image=nginx nginx-app --port=80 --env="DOMAIN=cluster" replicationcontroller "nginx-app" created # expose a port through with a service $ kubectl expose rc nginx-app --port=80 --name=nginx-http -{% endraw %} + {% endhighlight %} -With kubectl, we create a [replication controller](replication-controller.html) which will make sure that N pods are running nginx (where N is the number of replicas stated in the spec, which defaults to 1). We also create a [service](services.html) with a selector that matches the replication controller's selector. See the [Quick start](quick-start.html) for more information. +With kubectl, we create a [replication controller](replication-controller) which will make sure that N pods are running nginx (where N is the number of replicas stated in the spec, which defaults to 1). We also create a [service](services) with a selector that matches the replication controller's selector. See the [Quick start](quick-start) for more information. By default images are run in the background, similar to `docker run -d ...`, if you want to run things in the foreground, use: {% highlight console %} -{% raw %} + kubectl run [-i] [--tty] --attach --image= -{% endraw %} + {% endhighlight %} Unlike `docker run ...`, if `--attach` is specified, we attach to `stdin`, `stdout` and `stderr`, there is no ability to control which streams are attached (`docker -a ...`). @@ -68,65 +50,65 @@ To destroy the replication controller (and it's pods) you need to run `kubectl #### docker ps -How do I list what is currently running? Checkout [kubectl get](kubectl/kubectl_get.html). +How do I list what is currently running? Checkout [kubectl get](kubectl/kubectl_get). With docker: {% highlight console %} -{% raw %} + $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES a9ec34d98787 nginx "nginx -g 'daemon of About an hour ago Up About an hour 0.0.0.0:80->80/tcp, 443/tcp nginx-app -{% endraw %} + {% endhighlight %} With kubectl: {% highlight console %} -{% raw %} + $ kubectl get po NAME READY STATUS RESTARTS AGE nginx-app-5jyvm 1/1 Running 0 1h -{% endraw %} + {% endhighlight %} #### docker attach -How do I attach to a process that is already running in a container? Checkout [kubectl attach](kubectl/kubectl_attach.html) +How do I attach to a process that is already running in a container? Checkout [kubectl attach](kubectl/kubectl_attach) With docker: {% highlight console %} -{% raw %} + $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES a9ec34d98787 nginx "nginx -g 'daemon of 8 minutes ago Up 8 minutes 0.0.0.0:80->80/tcp, 443/tcp nginx-app $ docker attach -it a9ec34d98787 ... -{% endraw %} + {% endhighlight %} With kubectl: {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY STATUS RESTARTS AGE nginx-app-5jyvm 1/1 Running 0 10m $ kubectl attach -it nginx-app-5jyvm ... -{% endraw %} + {% endhighlight %} #### docker exec -How do I execute a command in a container? Checkout [kubectl exec](kubectl/kubectl_exec.html). +How do I execute a command in a container? Checkout [kubectl exec](kubectl/kubectl_exec). With docker: {% highlight console %} -{% raw %} + $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES @@ -134,13 +116,13 @@ a9ec34d98787 nginx "nginx -g 'daemon of 8 minutes ago $ docker exec a9ec34d98787 cat /etc/hostname a9ec34d98787 -{% endraw %} + {% endhighlight %} With kubectl: {% highlight console %} -{% raw %} + $ kubectl get po NAME READY STATUS RESTARTS AGE @@ -148,7 +130,7 @@ nginx-app-5jyvm 1/1 Running 0 10m $ kubectl exec nginx-app-5jyvm -- cat /etc/hostname nginx-app-5jyvm -{% endraw %} + {% endhighlight %} What about interactive commands? @@ -157,80 +139,80 @@ What about interactive commands? With docker: {% highlight console %} -{% raw %} + $ docker exec -ti a9ec34d98787 /bin/sh # exit -{% endraw %} + {% endhighlight %} With kubectl: {% highlight console %} -{% raw %} + $ kubectl exec -ti nginx-app-5jyvm -- /bin/sh # exit -{% endraw %} + {% endhighlight %} -For more information see [Getting into containers](getting-into-containers.html). +For more information see [Getting into containers](getting-into-containers). #### docker logs -How do I follow stdout/stderr of a running process? Checkout [kubectl logs](kubectl/kubectl_logs.html). +How do I follow stdout/stderr of a running process? Checkout [kubectl logs](kubectl/kubectl_logs). With docker: {% highlight console %} -{% raw %} + $ docker logs -f a9e 192.168.9.1 - - [14/Jul/2015:01:04:02 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.35.0" "-" 192.168.9.1 - - [14/Jul/2015:01:04:03 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.35.0" "-" -{% endraw %} + {% endhighlight %} With kubectl: {% highlight console %} -{% raw %} + $ kubectl logs -f nginx-app-zibvs 10.240.63.110 - - [14/Jul/2015:01:09:01 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.26.0" "-" 10.240.63.110 - - [14/Jul/2015:01:09:02 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.26.0" "-" -{% endraw %} + {% endhighlight %} Now's a good time to mention slight difference between pods and containers; by default pods will not terminate if their processes exit. Instead it will restart the process. This is similar to the docker run option `--restart=always` with one major difference. In docker, the output for each invocation of the process is concatenated but for Kubernetes, each invocation is separate. To see the output from a previous run in Kubernetes, do this: {% highlight console %} -{% raw %} + $ kubectl logs --previous nginx-app-zibvs 10.240.63.110 - - [14/Jul/2015:01:09:01 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.26.0" "-" 10.240.63.110 - - [14/Jul/2015:01:09:02 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.26.0" "-" -{% endraw %} + {% endhighlight %} -See [Logging](logging.html) for more information. +See [Logging](logging) for more information. #### docker stop and docker rm -How do I stop and delete a running process? Checkout [kubectl delete](kubectl/kubectl_delete.html). +How do I stop and delete a running process? Checkout [kubectl delete](kubectl/kubectl_delete). With docker {% highlight console %} -{% raw %} + $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES @@ -240,13 +222,13 @@ a9ec34d98787 $ docker rm a9ec34d98787 a9ec34d98787 -{% endraw %} + {% endhighlight %} With kubectl: {% highlight console %} -{% raw %} + $ kubectl get rc nginx-app CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS @@ -260,7 +242,7 @@ nginx-app-aualv 1/1 Running 0 16s $ kubectl get po NAME READY STATUS RESTARTS AGE -{% endraw %} + {% endhighlight %} Notice that we don't delete the pod directly. With kubectl we want to delete the replication controller that owns the pod. If we delete the pod directly, the replication controller will recreate the pod. @@ -271,12 +253,12 @@ There is no direct analog of `docker login` in kubectl. If you are interested in #### docker version -How do I get the version of my client and server? Checkout [kubectl version](kubectl/kubectl_version.html). +How do I get the version of my client and server? Checkout [kubectl version](kubectl/kubectl_version). With docker: {% highlight console %} -{% raw %} + $ docker version Client version: 1.7.0 @@ -290,29 +272,29 @@ Go version (server): go1.4.2 Git commit (server): 0baf609 OS/Arch (server): linux/amd64 -{% endraw %} + {% endhighlight %} With kubectl: {% highlight console %} -{% raw %} + $ kubectl version Client Version: version.Info{Major:"0", Minor:"20.1", GitVersion:"v0.20.1", GitCommit:"", GitTreeState:"not a git tree"} Server Version: version.Info{Major:"0", Minor:"21+", GitVersion:"v0.21.1-411-g32699e873ae1ca-dirty", GitCommit:"32699e873ae1caa01812e41de7eab28df4358ee4", GitTreeState:"dirty"} -{% endraw %} + {% endhighlight %} #### docker info -How do I get miscellaneous info about my environment and configuration? Checkout [kubectl cluster-info](kubectl/kubectl_cluster-info.html). +How do I get miscellaneous info about my environment and configuration? Checkout [kubectl cluster-info](kubectl/kubectl_cluster-info). With docker: {% highlight console %} -{% raw %} + $ docker info Containers: 40 @@ -332,13 +314,13 @@ Name: k8s-is-fun.mtv.corp.google.com ID: ADUV:GCYR:B3VJ:HMPO:LNPQ:KD5S:YKFQ:76VN:IANZ:7TFV:ZBF4:BYJO WARNING: No swap limit support -{% endraw %} + {% endhighlight %} With kubectl: {% highlight console %} -{% raw %} + $ kubectl cluster-info Kubernetes master is running at https://108.59.85.141 @@ -348,7 +330,7 @@ Grafana is running at https://108.59.85.141/api/v1/proxy/namespaces/kube-system/ Heapster is running at https://108.59.85.141/api/v1/proxy/namespaces/kube-system/services/monitoring-heapster InfluxDB is running at https://108.59.85.141/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb -{% endraw %} + {% endhighlight %} diff --git a/v1.1/docs/user-guide/downward-api.md b/v1.1/docs/user-guide/downward-api.md index 74d2323a73..8953a189ec 100644 --- a/v1.1/docs/user-guide/downward-api.md +++ b/v1.1/docs/user-guide/downward-api.md @@ -1,10 +1,6 @@ --- title: "Downward API" --- - - -# Downward API - It is sometimes useful for a container to have information about itself, but we want to be careful not to over-couple containers to Kubernetes. The downward API allows containers to consume information about themselves or the system and @@ -55,7 +51,7 @@ downward API: {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: @@ -79,7 +75,7 @@ spec: fieldRef: fieldPath: status.podIP restartPolicy: Never -{% endraw %} + {% endhighlight %} [Download example](downward-api/dapi-pod.yaml) @@ -93,13 +89,13 @@ Using a similar syntax it's possible to expose pod information to containers usi Downward API are dumped to a mounted volume. This is achieved using a `downwardAPI` volume type and the different items represent the files to be created. `fieldPath` references the field to be exposed. -Downward API volume permits to store more complex data like [`metadata.labels`](labels.html) and [`metadata.annotations`](annotations.html). Currently key/value pair set fields are saved using `key="value"` format: +Downward API volume permits to store more complex data like [`metadata.labels`](labels) and [`metadata.annotations`](annotations). Currently key/value pair set fields are saved using `key="value"` format: ``` -{% raw %} + key1="value1" key2="value2" -{% endraw %} + ``` In future, it will be possible to specify an output format option. @@ -122,7 +118,7 @@ This is an example of a pod that consumes its labels and annotations via the dow {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: @@ -153,7 +149,7 @@ spec: - path: "annotations" fieldRef: fieldPath: metadata.annotations -{% endraw %} + {% endhighlight %} [Download example](downward-api/volume/dapi-volume.yaml) diff --git a/v1.1/docs/user-guide/downward-api/README.md b/v1.1/docs/user-guide/downward-api/README.md index df2d46a1c6..66544df419 100644 --- a/v1.1/docs/user-guide/downward-api/README.md +++ b/v1.1/docs/user-guide/downward-api/README.md @@ -1,18 +1,14 @@ --- title: "Downward API example" --- - - -# Downward API example - Following this example, you will create a pod with a container that consumes the pod's name and -namespace using the [downward API](../downward-api.html). +namespace using the [downward API](../downward-api). ## Step Zero: Prerequisites This example assumes you have a Kubernetes cluster installed and running, and that you have installed the `kubectl` command line tool somewhere in your path. Please see the [getting -started](../../../docs/getting-started-guides/) for installation instructions for your platform. +started](/{{page.version}}/docs/getting-started-guides/) for installation instructions for your platform. ## Step One: Create the pod @@ -23,9 +19,9 @@ Use the [`examples/downward-api/dapi-pod.yaml`](dapi-pod.yaml) file to create a downward API. {% highlight console %} -{% raw %} + $ kubectl create -f docs/user-guide/downward-api/dapi-pod.yaml -{% endraw %} + {% endhighlight %} ### Examine the logs @@ -34,12 +30,12 @@ This pod runs the `env` command in a container that consumes the downward API. through the pod logs to see that the pod was injected with the correct values: {% highlight console %} -{% raw %} + $ kubectl logs dapi-test-pod | grep POD_ 2015-04-30T20:22:18.568024817Z MY_POD_NAME=dapi-test-pod 2015-04-30T20:22:18.568087688Z MY_POD_NAMESPACE=default 2015-04-30T20:22:18.568092435Z MY_POD_IP=10.0.1.6 -{% endraw %} + {% endhighlight %} diff --git a/v1.1/docs/user-guide/downward-api/index.md b/v1.1/docs/user-guide/downward-api/index.md index df2d46a1c6..66544df419 100644 --- a/v1.1/docs/user-guide/downward-api/index.md +++ b/v1.1/docs/user-guide/downward-api/index.md @@ -1,18 +1,14 @@ --- title: "Downward API example" --- - - -# Downward API example - Following this example, you will create a pod with a container that consumes the pod's name and -namespace using the [downward API](../downward-api.html). +namespace using the [downward API](../downward-api). ## Step Zero: Prerequisites This example assumes you have a Kubernetes cluster installed and running, and that you have installed the `kubectl` command line tool somewhere in your path. Please see the [getting -started](../../../docs/getting-started-guides/) for installation instructions for your platform. +started](/{{page.version}}/docs/getting-started-guides/) for installation instructions for your platform. ## Step One: Create the pod @@ -23,9 +19,9 @@ Use the [`examples/downward-api/dapi-pod.yaml`](dapi-pod.yaml) file to create a downward API. {% highlight console %} -{% raw %} + $ kubectl create -f docs/user-guide/downward-api/dapi-pod.yaml -{% endraw %} + {% endhighlight %} ### Examine the logs @@ -34,12 +30,12 @@ This pod runs the `env` command in a container that consumes the downward API. through the pod logs to see that the pod was injected with the correct values: {% highlight console %} -{% raw %} + $ kubectl logs dapi-test-pod | grep POD_ 2015-04-30T20:22:18.568024817Z MY_POD_NAME=dapi-test-pod 2015-04-30T20:22:18.568087688Z MY_POD_NAMESPACE=default 2015-04-30T20:22:18.568092435Z MY_POD_IP=10.0.1.6 -{% endraw %} + {% endhighlight %} diff --git a/v1.1/docs/user-guide/downward-api/volume/README.md b/v1.1/docs/user-guide/downward-api/volume/README.md index 272b2e141b..5a67a602ea 100644 --- a/v1.1/docs/user-guide/downward-api/volume/README.md +++ b/v1.1/docs/user-guide/downward-api/volume/README.md @@ -1,12 +1,8 @@ --- title: "Downward API volume plugin" --- - - -# Downward API volume plugin - Following this example, you will create a pod with a downward API volume. -A downward API volume is a k8s volume plugin with the ability to save some pod information in a plain text file. The pod information can be for example some [metadata](../../../../docs/devel/api-conventions.html#metadata). +A downward API volume is a k8s volume plugin with the ability to save some pod information in a plain text file. The pod information can be for example some [metadata](..//{{page.version}}/docs/devel/api-conventions.html#metadata). Supported metadata fields: @@ -17,16 +13,16 @@ Supported metadata fields: ### Step Zero: Prerequisites -This example assumes you have a Kubernetes cluster installed and running, and the ```kubectl``` command line tool somewhere in your path. Please see the [gettingstarted](../../../../docs/getting-started-guides/) for installation instructions for your platform. +This example assumes you have a Kubernetes cluster installed and running, and the ```kubectl``` command line tool somewhere in your path. Please see the [gettingstarted](..//{{page.version}}/docs/getting-started-guides/) for installation instructions for your platform. ### Step One: Create the pod Use the `docs/user-guide/downward-api/dapi-volume.yaml` file to create a Pod with a  downward API volume which stores pod labels and pod annotations to `/etc/labels` and  `/etc/annotations` respectively. {% highlight sh %} -{% raw %} + $ kubectl create -f docs/user-guide/downward-api/volume/dapi-volume.yaml -{% endraw %} + {% endhighlight %} ### Step Two: Examine pod/container output @@ -34,7 +30,7 @@ $ kubectl create -f docs/user-guide/downward-api/volume/dapi-volume.yaml The pod displays (every 5 seconds) the content of the dump files which can be executed via the usual `kubectl log` command {% highlight sh %} -{% raw %} + $ kubectl logs kubernetes-downwardapi-volume-example cluster="test-cluster1" rack="rack-22" @@ -43,7 +39,7 @@ build="two" builder="john-doe" kubernetes.io/config.seen="2015-08-24T13:47:23.432459138Z" kubernetes.io/config.source="api" -{% endraw %} + {% endhighlight %} ### Internals @@ -51,7 +47,7 @@ kubernetes.io/config.source="api" In pod's `/etc` directory one may find the file created by the plugin (system files elided): {% highlight sh %} -{% raw %} + $ kubectl exec kubernetes-downwardapi-volume-example -i -t -- sh / # ls -laR /etc /etc: @@ -70,7 +66,7 @@ drwxrwxrwt 3 0 0 180 Aug 24 13:03 .. -rw-r--r-- 1 0 0 115 Aug 24 13:03 annotations -rw-r--r-- 1 0 0 53 Aug 24 13:03 labels / # -{% endraw %} + {% endhighlight %} The file `labels` is stored in a temporary directory (`..2015_08_24_13_03_44259413923` in the example above) which is symlinked to by `..downwardapi`. Symlinks for annotations and labels in `/etc` point to files containing the actual metadata through the `..downwardapi` indirection.  This structure allows for dynamic atomic refresh of the metadata: updates are written to a new temporary directory, and the `..downwardapi` symlink is updated atomically using `rename(2)`. diff --git a/v1.1/docs/user-guide/downward-api/volume/index.md b/v1.1/docs/user-guide/downward-api/volume/index.md index 272b2e141b..5a67a602ea 100644 --- a/v1.1/docs/user-guide/downward-api/volume/index.md +++ b/v1.1/docs/user-guide/downward-api/volume/index.md @@ -1,12 +1,8 @@ --- title: "Downward API volume plugin" --- - - -# Downward API volume plugin - Following this example, you will create a pod with a downward API volume. -A downward API volume is a k8s volume plugin with the ability to save some pod information in a plain text file. The pod information can be for example some [metadata](../../../../docs/devel/api-conventions.html#metadata). +A downward API volume is a k8s volume plugin with the ability to save some pod information in a plain text file. The pod information can be for example some [metadata](..//{{page.version}}/docs/devel/api-conventions.html#metadata). Supported metadata fields: @@ -17,16 +13,16 @@ Supported metadata fields: ### Step Zero: Prerequisites -This example assumes you have a Kubernetes cluster installed and running, and the ```kubectl``` command line tool somewhere in your path. Please see the [gettingstarted](../../../../docs/getting-started-guides/) for installation instructions for your platform. +This example assumes you have a Kubernetes cluster installed and running, and the ```kubectl``` command line tool somewhere in your path. Please see the [gettingstarted](..//{{page.version}}/docs/getting-started-guides/) for installation instructions for your platform. ### Step One: Create the pod Use the `docs/user-guide/downward-api/dapi-volume.yaml` file to create a Pod with a  downward API volume which stores pod labels and pod annotations to `/etc/labels` and  `/etc/annotations` respectively. {% highlight sh %} -{% raw %} + $ kubectl create -f docs/user-guide/downward-api/volume/dapi-volume.yaml -{% endraw %} + {% endhighlight %} ### Step Two: Examine pod/container output @@ -34,7 +30,7 @@ $ kubectl create -f docs/user-guide/downward-api/volume/dapi-volume.yaml The pod displays (every 5 seconds) the content of the dump files which can be executed via the usual `kubectl log` command {% highlight sh %} -{% raw %} + $ kubectl logs kubernetes-downwardapi-volume-example cluster="test-cluster1" rack="rack-22" @@ -43,7 +39,7 @@ build="two" builder="john-doe" kubernetes.io/config.seen="2015-08-24T13:47:23.432459138Z" kubernetes.io/config.source="api" -{% endraw %} + {% endhighlight %} ### Internals @@ -51,7 +47,7 @@ kubernetes.io/config.source="api" In pod's `/etc` directory one may find the file created by the plugin (system files elided): {% highlight sh %} -{% raw %} + $ kubectl exec kubernetes-downwardapi-volume-example -i -t -- sh / # ls -laR /etc /etc: @@ -70,7 +66,7 @@ drwxrwxrwt 3 0 0 180 Aug 24 13:03 .. -rw-r--r-- 1 0 0 115 Aug 24 13:03 annotations -rw-r--r-- 1 0 0 53 Aug 24 13:03 labels / # -{% endraw %} + {% endhighlight %} The file `labels` is stored in a temporary directory (`..2015_08_24_13_03_44259413923` in the example above) which is symlinked to by `..downwardapi`. Symlinks for annotations and labels in `/etc` point to files containing the actual metadata through the `..downwardapi` indirection.  This structure allows for dynamic atomic refresh of the metadata: updates are written to a new temporary directory, and the `..downwardapi` symlink is updated atomically using `rename(2)`. diff --git a/v1.1/docs/user-guide/environment-guide/README.md b/v1.1/docs/user-guide/environment-guide/README.md index 723c879029..112e2d43b2 100644 --- a/v1.1/docs/user-guide/environment-guide/README.md +++ b/v1.1/docs/user-guide/environment-guide/README.md @@ -1,98 +1,96 @@ ---- -title: "Environment Guide Example" ---- - -Environment Guide Example -========================= -This example demonstrates running pods, replication controllers, and -services. It shows two types of pods: frontend and backend, with -services on top of both. Accessing the frontend pod will return -environment information about itself, and a backend pod that it has -accessed through the service. The goal is to illuminate the -environment metadata available to running containers inside the -Kubernetes cluster. The documentation for the Kubernetes environment -is [here](../../../docs/user-guide/container-environment.html). - -![Diagram](diagram.png) - -Prerequisites -------------- -This example assumes that you have a Kubernetes cluster installed and -running, and that you have installed the `kubectl` command line tool -somewhere in your path. Please see the [getting -started](../../../docs/getting-started-guides/) for installation instructions -for your platform. - -Optional: Build your own containers ------------------------------------ -The code for the containers is under -[containers/](containers/) - -Get everything running ----------------------- - - kubectl create -f ./backend-rc.yaml - kubectl create -f ./backend-srv.yaml - kubectl create -f ./show-rc.yaml - kubectl create -f ./show-srv.yaml - -Query the service ------------------ -Use `kubectl describe service show-srv` to determine the public IP of -your service. - -> Note: If your platform does not support external load balancers, - you'll need to open the proper port and direct traffic to the - internal IP shown for the frontend service with the above command - -Run `curl :80` to query the service. You should get -something like this back: - -``` -{% raw %} -Pod Name: show-rc-xxu6i -Pod Namespace: default -USER_VAR: important information - -Kubernetes environment variables -BACKEND_SRV_SERVICE_HOST = 10.147.252.185 -BACKEND_SRV_SERVICE_PORT = 5000 -KUBERNETES_RO_SERVICE_HOST = 10.147.240.1 -KUBERNETES_RO_SERVICE_PORT = 80 -KUBERNETES_SERVICE_HOST = 10.147.240.2 -KUBERNETES_SERVICE_PORT = 443 -KUBE_DNS_SERVICE_HOST = 10.147.240.10 -KUBE_DNS_SERVICE_PORT = 53 - -Found backend ip: 10.147.252.185 port: 5000 -Response from backend -Backend Container -Backend Pod Name: backend-rc-6qiya -Backend Namespace: default -{% endraw %} -``` - -First the frontend pod's information is printed. The pod name and -[namespace](../../../docs/design/namespaces.html) are retrieved from the -[Downward API](../../../docs/user-guide/downward-api.html). Next, `USER_VAR` is the name of -an environment variable set in the [pod -definition](show-rc.yaml). Then, the dynamic Kubernetes environment -variables are scanned and printed. These are used to find the backend -service, named `backend-srv`. Finally, the frontend pod queries the -backend service and prints the information returned. Again the backend -pod returns its own pod name and namespace. - -Try running the `curl` command a few times, and notice what -changes. Ex: `watch -n 1 curl -s ` Firstly, the frontend service -is directing your request to different frontend pods each time. The -frontend pods are always contacting the backend through the backend -service. This results in a different backend pod servicing each -request as well. - -Cleanup -------- - kubectl delete rc,service -l type=show-type - kubectl delete rc,service -l type=backend-type - - - +--- +title: "Environment Guide Example" +--- + +Environment Guide Example +========================= +This example demonstrates running pods, replication controllers, and +services. It shows two types of pods: frontend and backend, with +services on top of both. Accessing the frontend pod will return +environment information about itself, and a backend pod that it has +accessed through the service. The goal is to illuminate the +environment metadata available to running containers inside the +Kubernetes cluster. The documentation for the Kubernetes environment +is [here](/{{page.version}}/docs/user-guide/container-environment). + +![Diagram](diagram.png) + +Prerequisites +------------- +This example assumes that you have a Kubernetes cluster installed and +running, and that you have installed the `kubectl` command line tool +somewhere in your path. Please see the [getting +started](/{{page.version}}/docs/getting-started-guides/) for installation instructions +for your platform. + +Optional: Build your own containers +----------------------------------- +The code for the containers is under +[containers/](containers/) + +Get everything running +---------------------- + + kubectl create -f ./backend-rc.yaml + kubectl create -f ./backend-srv.yaml + kubectl create -f ./show-rc.yaml + kubectl create -f ./show-srv.yaml + +Query the service +----------------- +Use `kubectl describe service show-srv` to determine the public IP of +your service. + +> Note: If your platform does not support external load balancers, + you'll need to open the proper port and direct traffic to the + internal IP shown for the frontend service with the above command + +Run `curl :80` to query the service. You should get +something like this back: + +``` +Pod Name: show-rc-xxu6i +Pod Namespace: default +USER_VAR: important information + +Kubernetes environment variables +BACKEND_SRV_SERVICE_HOST = 10.147.252.185 +BACKEND_SRV_SERVICE_PORT = 5000 +KUBERNETES_RO_SERVICE_HOST = 10.147.240.1 +KUBERNETES_RO_SERVICE_PORT = 80 +KUBERNETES_SERVICE_HOST = 10.147.240.2 +KUBERNETES_SERVICE_PORT = 443 +KUBE_DNS_SERVICE_HOST = 10.147.240.10 +KUBE_DNS_SERVICE_PORT = 53 + +Found backend ip: 10.147.252.185 port: 5000 +Response from backend +Backend Container +Backend Pod Name: backend-rc-6qiya +Backend Namespace: default +``` + +First the frontend pod's information is printed. The pod name and +[namespace](/{{page.version}}/docs/design/namespaces) are retrieved from the +[Downward API](/{{page.version}}/docs/user-guide/downward-api). Next, `USER_VAR` is the name of +an environment variable set in the [pod +definition](show-rc.yaml). Then, the dynamic Kubernetes environment +variables are scanned and printed. These are used to find the backend +service, named `backend-srv`. Finally, the frontend pod queries the +backend service and prints the information returned. Again the backend +pod returns its own pod name and namespace. + +Try running the `curl` command a few times, and notice what +changes. Ex: `watch -n 1 curl -s ` Firstly, the frontend service +is directing your request to different frontend pods each time. The +frontend pods are always contacting the backend through the backend +service. This results in a different backend pod servicing each +request as well. + +Cleanup +------- + kubectl delete rc,service -l type=show-type + kubectl delete rc,service -l type=backend-type + + + diff --git a/v1.1/docs/user-guide/environment-guide/index.md b/v1.1/docs/user-guide/environment-guide/index.md index 723c879029..112e2d43b2 100644 --- a/v1.1/docs/user-guide/environment-guide/index.md +++ b/v1.1/docs/user-guide/environment-guide/index.md @@ -1,98 +1,96 @@ ---- -title: "Environment Guide Example" ---- - -Environment Guide Example -========================= -This example demonstrates running pods, replication controllers, and -services. It shows two types of pods: frontend and backend, with -services on top of both. Accessing the frontend pod will return -environment information about itself, and a backend pod that it has -accessed through the service. The goal is to illuminate the -environment metadata available to running containers inside the -Kubernetes cluster. The documentation for the Kubernetes environment -is [here](../../../docs/user-guide/container-environment.html). - -![Diagram](diagram.png) - -Prerequisites -------------- -This example assumes that you have a Kubernetes cluster installed and -running, and that you have installed the `kubectl` command line tool -somewhere in your path. Please see the [getting -started](../../../docs/getting-started-guides/) for installation instructions -for your platform. - -Optional: Build your own containers ------------------------------------ -The code for the containers is under -[containers/](containers/) - -Get everything running ----------------------- - - kubectl create -f ./backend-rc.yaml - kubectl create -f ./backend-srv.yaml - kubectl create -f ./show-rc.yaml - kubectl create -f ./show-srv.yaml - -Query the service ------------------ -Use `kubectl describe service show-srv` to determine the public IP of -your service. - -> Note: If your platform does not support external load balancers, - you'll need to open the proper port and direct traffic to the - internal IP shown for the frontend service with the above command - -Run `curl :80` to query the service. You should get -something like this back: - -``` -{% raw %} -Pod Name: show-rc-xxu6i -Pod Namespace: default -USER_VAR: important information - -Kubernetes environment variables -BACKEND_SRV_SERVICE_HOST = 10.147.252.185 -BACKEND_SRV_SERVICE_PORT = 5000 -KUBERNETES_RO_SERVICE_HOST = 10.147.240.1 -KUBERNETES_RO_SERVICE_PORT = 80 -KUBERNETES_SERVICE_HOST = 10.147.240.2 -KUBERNETES_SERVICE_PORT = 443 -KUBE_DNS_SERVICE_HOST = 10.147.240.10 -KUBE_DNS_SERVICE_PORT = 53 - -Found backend ip: 10.147.252.185 port: 5000 -Response from backend -Backend Container -Backend Pod Name: backend-rc-6qiya -Backend Namespace: default -{% endraw %} -``` - -First the frontend pod's information is printed. The pod name and -[namespace](../../../docs/design/namespaces.html) are retrieved from the -[Downward API](../../../docs/user-guide/downward-api.html). Next, `USER_VAR` is the name of -an environment variable set in the [pod -definition](show-rc.yaml). Then, the dynamic Kubernetes environment -variables are scanned and printed. These are used to find the backend -service, named `backend-srv`. Finally, the frontend pod queries the -backend service and prints the information returned. Again the backend -pod returns its own pod name and namespace. - -Try running the `curl` command a few times, and notice what -changes. Ex: `watch -n 1 curl -s ` Firstly, the frontend service -is directing your request to different frontend pods each time. The -frontend pods are always contacting the backend through the backend -service. This results in a different backend pod servicing each -request as well. - -Cleanup -------- - kubectl delete rc,service -l type=show-type - kubectl delete rc,service -l type=backend-type - - - +--- +title: "Environment Guide Example" +--- + +Environment Guide Example +========================= +This example demonstrates running pods, replication controllers, and +services. It shows two types of pods: frontend and backend, with +services on top of both. Accessing the frontend pod will return +environment information about itself, and a backend pod that it has +accessed through the service. The goal is to illuminate the +environment metadata available to running containers inside the +Kubernetes cluster. The documentation for the Kubernetes environment +is [here](/{{page.version}}/docs/user-guide/container-environment). + +![Diagram](diagram.png) + +Prerequisites +------------- +This example assumes that you have a Kubernetes cluster installed and +running, and that you have installed the `kubectl` command line tool +somewhere in your path. Please see the [getting +started](/{{page.version}}/docs/getting-started-guides/) for installation instructions +for your platform. + +Optional: Build your own containers +----------------------------------- +The code for the containers is under +[containers/](containers/) + +Get everything running +---------------------- + + kubectl create -f ./backend-rc.yaml + kubectl create -f ./backend-srv.yaml + kubectl create -f ./show-rc.yaml + kubectl create -f ./show-srv.yaml + +Query the service +----------------- +Use `kubectl describe service show-srv` to determine the public IP of +your service. + +> Note: If your platform does not support external load balancers, + you'll need to open the proper port and direct traffic to the + internal IP shown for the frontend service with the above command + +Run `curl :80` to query the service. You should get +something like this back: + +``` +Pod Name: show-rc-xxu6i +Pod Namespace: default +USER_VAR: important information + +Kubernetes environment variables +BACKEND_SRV_SERVICE_HOST = 10.147.252.185 +BACKEND_SRV_SERVICE_PORT = 5000 +KUBERNETES_RO_SERVICE_HOST = 10.147.240.1 +KUBERNETES_RO_SERVICE_PORT = 80 +KUBERNETES_SERVICE_HOST = 10.147.240.2 +KUBERNETES_SERVICE_PORT = 443 +KUBE_DNS_SERVICE_HOST = 10.147.240.10 +KUBE_DNS_SERVICE_PORT = 53 + +Found backend ip: 10.147.252.185 port: 5000 +Response from backend +Backend Container +Backend Pod Name: backend-rc-6qiya +Backend Namespace: default +``` + +First the frontend pod's information is printed. The pod name and +[namespace](/{{page.version}}/docs/design/namespaces) are retrieved from the +[Downward API](/{{page.version}}/docs/user-guide/downward-api). Next, `USER_VAR` is the name of +an environment variable set in the [pod +definition](show-rc.yaml). Then, the dynamic Kubernetes environment +variables are scanned and printed. These are used to find the backend +service, named `backend-srv`. Finally, the frontend pod queries the +backend service and prints the information returned. Again the backend +pod returns its own pod name and namespace. + +Try running the `curl` command a few times, and notice what +changes. Ex: `watch -n 1 curl -s ` Firstly, the frontend service +is directing your request to different frontend pods each time. The +frontend pods are always contacting the backend through the backend +service. This results in a different backend pod servicing each +request as well. + +Cleanup +------- + kubectl delete rc,service -l type=show-type + kubectl delete rc,service -l type=backend-type + + + diff --git a/v1.1/docs/user-guide/getting-into-containers.md b/v1.1/docs/user-guide/getting-into-containers.md index 407d4eefdf..dfde0cdc2b 100644 --- a/v1.1/docs/user-guide/getting-into-containers.md +++ b/v1.1/docs/user-guide/getting-into-containers.md @@ -1,87 +1,72 @@ ---- -title: "Getting into containers: kubectl exec" -section: support ---- -Developers can use `kubectl exec` to run commands in a container. This guide demonstrates two use cases. - -## Using kubectl exec to check the environment variables of a container - -Kubernetes exposes [services](services.html#environment-variables) through environment variables. It is convenient to check these environment variables using `kubectl exec`. - - -We first create a pod and a service, - -{% highlight console %} -{% raw %} -$ kubectl create -f examples/guestbook/redis-master-controller.yaml -$ kubectl create -f examples/guestbook/redis-master-service.yaml -{% endraw %} -{% endhighlight %} - -wait until the pod is Running and Ready, - -{% highlight console %} -{% raw %} -$ kubectl get pod -NAME READY REASON RESTARTS AGE -redis-master-ft9ex 1/1 Running 0 12s -{% endraw %} -{% endhighlight %} - -then we can check the environment variables of the pod, - -{% highlight console %} -{% raw %} -$ kubectl exec redis-master-ft9ex env -... -REDIS_MASTER_SERVICE_PORT=6379 -REDIS_MASTER_SERVICE_HOST=10.0.0.219 -... -{% endraw %} -{% endhighlight %} - -We can use these environment variables in applications to find the service. - - -## Using kubectl exec to check the mounted volumes - -It is convenient to use `kubectl exec` to check if the volumes are mounted as expected. -We first create a Pod with a volume mounted at /data/redis, - -{% highlight console %} -{% raw %} -kubectl create -f docs/user-guide/walkthrough/pod-redis.yaml -{% endraw %} -{% endhighlight %} - -wait until the pod is Running and Ready, - -{% highlight console %} -{% raw %} -$ kubectl get pods -NAME READY REASON RESTARTS AGE -storage 1/1 Running 0 1m -{% endraw %} -{% endhighlight %} - -we then use `kubectl exec` to verify that the volume is mounted at /data/redis, - -{% highlight console %} -{% raw %} -$ kubectl exec storage ls /data -redis -{% endraw %} -{% endhighlight %} - -## Using kubectl exec to open a bash terminal in a pod - -After all, open a terminal in a pod is the most direct way to introspect the pod. Assuming the pod/storage is still running, run - -{% highlight console %} -{% raw %} -$ kubectl exec -ti storage -- bash -root@storage:/data# -{% endraw %} -{% endhighlight %} - +--- +title: "Getting into containers: kubectl exec" +--- +Developers can use `kubectl exec` to run commands in a container. This guide demonstrates two use cases. + +## Using kubectl exec to check the environment variables of a container + +Kubernetes exposes [services](services.html#environment-variables) through environment variables. It is convenient to check these environment variables using `kubectl exec`. + + +We first create a pod and a service, + +{% highlight console %} +$ kubectl create -f examples/guestbook/redis-master-controller.yaml +$ kubectl create -f examples/guestbook/redis-master-service.yaml +{% endhighlight %} + +wait until the pod is Running and Ready, + +{% highlight console %} +$ kubectl get pod +NAME READY REASON RESTARTS AGE +redis-master-ft9ex 1/1 Running 0 12s +{% endhighlight %} + +then we can check the environment variables of the pod, + +{% highlight console %} +$ kubectl exec redis-master-ft9ex env +... +REDIS_MASTER_SERVICE_PORT=6379 +REDIS_MASTER_SERVICE_HOST=10.0.0.219 +... +{% endhighlight %} + +We can use these environment variables in applications to find the service. + + +## Using kubectl exec to check the mounted volumes + +It is convenient to use `kubectl exec` to check if the volumes are mounted as expected. +We first create a Pod with a volume mounted at /data/redis, + +{% highlight console %} +kubectl create -f docs/user-guide/walkthrough/pod-redis.yaml +{% endhighlight %} + +wait until the pod is Running and Ready, + +{% highlight console %} +$ kubectl get pods +NAME READY REASON RESTARTS AGE +storage 1/1 Running 0 1m +{% endhighlight %} + +we then use `kubectl exec` to verify that the volume is mounted at /data/redis, + +{% highlight console %} +$ kubectl exec storage ls /data +redis +{% endhighlight %} + +## Using kubectl exec to open a bash terminal in a pod + +After all, open a terminal in a pod is the most direct way to introspect the pod. Assuming the pod/storage is still running, run + +{% highlight console %} +$ kubectl exec -ti storage -- bash +root@storage:/data# +{% endhighlight %} + This gets you a terminal. \ No newline at end of file diff --git a/v1.1/docs/user-guide/horizontal-pod-autoscaler.md b/v1.1/docs/user-guide/horizontal-pod-autoscaler.md index 952e799108..fc79d4c5c7 100644 --- a/v1.1/docs/user-guide/horizontal-pod-autoscaler.md +++ b/v1.1/docs/user-guide/horizontal-pod-autoscaler.md @@ -1,25 +1,8 @@ --- title: "Horizontal Pod Autoscaler" --- - - -# Horizontal Pod Autoscaler - This document describes the current state of Horizontal Pod Autoscaler in Kubernetes. - - -- [Horizontal Pod Autoscaler](#horizontal-pod-autoscaler) - - [What is Horizontal Pod Autoscaler?](#what-is-horizontal-pod-autoscaler) - - [How does Horizontal Pod Autoscaler work?](#how-does-horizontal-pod-autoscaler-work) - - [API Object](#api-object) - - [Support for horizontal pod autoscaler in kubectl](#support-for-horizontal-pod-autoscaler-in-kubectl) - - [Autoscaling during rolling update](#autoscaling-during-rolling-update) - - [Further reading](#further-reading) - - - - ## What is Horizontal Pod Autoscaler? Horizontal pod autoscaling allows the number of pods in a replication controller or deployment @@ -72,7 +55,7 @@ In addition, there is a special `kubectl autoscale` command that allows for easy For instance, executing `kubectl autoscale rc foo --min=2 --max=5 --cpu-percent=80` will create an autoscaler for replication controller *foo*, with target CPU utilization set to `80%` and the number of replicas between 2 and 5. -The detailed documentation of `kubectl autoscale` can be found [here](kubectl/kubectl_autoscale.html). +The detailed documentation of `kubectl autoscale` can be found [here](kubectl/kubectl_autoscale). ## Autoscaling during rolling update @@ -90,9 +73,9 @@ the horizontal pod autoscaler will not be bound to the new replication controlle ## Further reading -* Design documentation: [Horizontal Pod Autoscaling](../design/horizontal-pod-autoscaler.html). -* Manual of autoscale command in kubectl: [kubectl autoscale](kubectl/kubectl_autoscale.html). -* Usage example of [Horizontal Pod Autoscaler](horizontal-pod-autoscaling/README.html). +* Design documentation: [Horizontal Pod Autoscaling](../design/horizontal-pod-autoscaler). +* Manual of autoscale command in kubectl: [kubectl autoscale](kubectl/kubectl_autoscale). +* Usage example of [Horizontal Pod Autoscaler](horizontal-pod-autoscaling/README). diff --git a/v1.1/docs/user-guide/horizontal-pod-autoscaling/README.md b/v1.1/docs/user-guide/horizontal-pod-autoscaling/README.md index ce1b3704f5..fcbfd2ffb4 100644 --- a/v1.1/docs/user-guide/horizontal-pod-autoscaling/README.md +++ b/v1.1/docs/user-guide/horizontal-pod-autoscaling/README.md @@ -1,11 +1,7 @@ --- title: "Horizontal Pod Autoscaler" --- - - -# Horizontal Pod Autoscaler - -Horizontal pod autoscaling is a [beta](../../../docs/api.html#api-versioning) feature in Kubernetes 1.1. +Horizontal pod autoscaling is a [beta](/{{page.version}}/docs/api.html#api-versioning) feature in Kubernetes 1.1. It allows the number of pods in a replication controller or deployment to scale automatically based on observed CPU usage. In the future also other metrics will be supported. @@ -16,7 +12,7 @@ In this document we explain how this feature works by walking you through an exa This example requires a running Kubernetes cluster and kubectl in the version at least 1.1. [Heapster](https://github.com/kubernetes/heapster) monitoring needs to be deployed in the cluster as horizontal pod autoscaler uses it to collect metrics -(if you followed [getting started on GCE guide](../../../docs/getting-started-guides/gce.html), +(if you followed [getting started on GCE guide](/{{page.version}}/docs/getting-started-guides/gce), heapster monitoring will be turned-on by default). @@ -31,42 +27,42 @@ First, we will start a replication controller running the image and expose it as {% highlight console %} -{% raw %} + $ kubectl run php-apache --image=gcr.io/google_containers/hpa-example --requests=cpu=200m replicationcontroller "php-apache" created $ kubectl expose rc php-apache --port=80 --type=LoadBalancer service "php-apache" exposed -{% endraw %} + {% endhighlight %} Now, we will wait some time and verify that both the replication controller and the service were correctly created and are running. We will also determine the IP address of the service: {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY STATUS RESTARTS AGE php-apache-wa3t1 1/1 Running 0 12m $ kubectl describe services php-apache | grep "LoadBalancer Ingress" LoadBalancer Ingress: 146.148.24.244 -{% endraw %} + {% endhighlight %} We may now check that php-apache server works correctly by calling ``curl`` with the service's IP: {% highlight console %} -{% raw %} + $ curl http://146.148.24.244 OK! -{% endraw %} + {% endhighlight %} Please notice that when exposing the service we assumed that our cluster runs on a provider which supports load balancers (e.g.: on GCE). If load balancers are not supported (e.g.: on Vagrant), we can expose php-apache service as ``ClusterIP`` and connect to it using the proxy on the master: {% highlight console %} -{% raw %} + $ kubectl expose rc php-apache --port=80 --type=ClusterIP service "php-apache" exposed @@ -75,7 +71,7 @@ Kubernetes master is running at https://146.148.6.215 $ curl -k -u : https://146.148.6.215/api/v1/proxy/namespaces/default/services/php-apache/ OK! -{% endraw %} + {% endhighlight %} @@ -85,7 +81,7 @@ Now that the server is running, we will create a horizontal pod autoscaler for i To create it, we will use the [hpa-php-apache.yaml](hpa-php-apache.yaml) file, which looks like this: {% highlight yaml %} -{% raw %} + apiVersion: extensions/v1beta1 kind: HorizontalPodAutoscaler metadata: @@ -100,7 +96,7 @@ spec: maxReplicas: 10 cpuUtilization: targetPercentage: 50 -{% endraw %} + {% endhighlight %} This defines a horizontal pod autoscaler that maintains between 1 and 10 replicas of the Pods @@ -108,35 +104,35 @@ controlled by the php-apache replication controller we created in the first step Roughly speaking, the horizontal autoscaler will increase and decrease the number of replicas (via the replication controller) so as to maintain an average CPU utilization across all Pods of 50% (since each pod requests 200 milli-cores by [kubectl run](#kubectl-run), this means average CPU utilization of 100 milli-cores). -See [here](../../../docs/design/horizontal-pod-autoscaler.html#autoscaling-algorithm) for more details on the algorithm. +See [here](/{{page.version}}/docs/design/horizontal-pod-autoscaler.html#autoscaling-algorithm) for more details on the algorithm. We will create the autoscaler by executing the following command: {% highlight console %} -{% raw %} + $ kubectl create -f docs/user-guide/horizontal-pod-autoscaling/hpa-php-apache.yaml horizontalpodautoscaler "php-apache" created -{% endraw %} + {% endhighlight %} -Alternatively, we can create the autoscaler using [kubectl autoscale](../kubectl/kubectl_autoscale.html). +Alternatively, we can create the autoscaler using [kubectl autoscale](../kubectl/kubectl_autoscale). The following command will create the equivalent autoscaler as defined in the [hpa-php-apache.yaml](hpa-php-apache.yaml) file: ``` -{% raw %} + $ kubectl autoscale rc php-apache --cpu-percent=50 --min=1 --max=10 replicationcontroller "php-apache" autoscaled -{% endraw %} + ``` We may check the current status of autoscaler by running: {% highlight console %} -{% raw %} + $ kubectl get hpa NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE php-apache ReplicationController/default/php-apache/ 50% 0% 1 10 27s -{% endraw %} + {% endhighlight %} Please note that the current CPU consumption is 0% as we are not sending any requests to the server @@ -148,44 +144,44 @@ Now, we will see how the autoscaler reacts on the increased load of the server. We will start an infinite loop of queries to our server (please run it in a different terminal): {% highlight console %} -{% raw %} + $ while true; do curl http://146.148.6.244; done -{% endraw %} + {% endhighlight %} We may examine, how CPU load was increased (the results should be visible after about 3-4 minutes) by executing: {% highlight console %} -{% raw %} + $ kubectl get hpa NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE php-apache ReplicationController/default/php-apache/ 50% 305% 1 10 4m -{% endraw %} + {% endhighlight %} In the case presented here, it bumped CPU consumption to 305% of the request. As a result, the replication controller was resized to 7 replicas: {% highlight console %} -{% raw %} + $ kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE php-apache php-apache gcr.io/google_containers/hpa-example run=php-apache 7 18m -{% endraw %} + {% endhighlight %} Now, we may increase the load even more by running yet another infinite loop of queries (in yet another terminal): {% highlight console %} -{% raw %} + $ while true; do curl http://146.148.6.244; done -{% endraw %} + {% endhighlight %} In the case presented here, it increased the number of serving pods to 10: {% highlight console %} -{% raw %} + $ kubectl get hpa NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE php-apache ReplicationController/default/php-apache/ 50% 65% 1 10 14m @@ -193,7 +189,7 @@ php-apache ReplicationController/default/php-apache/ 50% 65% 1 $ kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE php-apache php-apache gcr.io/google_containers/hpa-example run=php-apache 10 24m -{% endraw %} + {% endhighlight %} ## Step Four: Stop load @@ -202,7 +198,7 @@ We will finish our example by stopping the user load. We will terminate both infinite ``while`` loops sending requests to the server and verify the result state: {% highlight console %} -{% raw %} + $ kubectl get hpa NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE php-apache ReplicationController/default/php-apache/ 50% 0% 1 10 21m @@ -210,7 +206,7 @@ php-apache ReplicationController/default/php-apache/ 50% 0% 1 $ kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE php-apache php-apache gcr.io/google_containers/hpa-example run=php-apache 1 31m -{% endraw %} + {% endhighlight %} As we see, in the presented case CPU utilization dropped to 0, and the number of replicas dropped to 1. diff --git a/v1.1/docs/user-guide/horizontal-pod-autoscaling/index.md b/v1.1/docs/user-guide/horizontal-pod-autoscaling/index.md index ce1b3704f5..fcbfd2ffb4 100644 --- a/v1.1/docs/user-guide/horizontal-pod-autoscaling/index.md +++ b/v1.1/docs/user-guide/horizontal-pod-autoscaling/index.md @@ -1,11 +1,7 @@ --- title: "Horizontal Pod Autoscaler" --- - - -# Horizontal Pod Autoscaler - -Horizontal pod autoscaling is a [beta](../../../docs/api.html#api-versioning) feature in Kubernetes 1.1. +Horizontal pod autoscaling is a [beta](/{{page.version}}/docs/api.html#api-versioning) feature in Kubernetes 1.1. It allows the number of pods in a replication controller or deployment to scale automatically based on observed CPU usage. In the future also other metrics will be supported. @@ -16,7 +12,7 @@ In this document we explain how this feature works by walking you through an exa This example requires a running Kubernetes cluster and kubectl in the version at least 1.1. [Heapster](https://github.com/kubernetes/heapster) monitoring needs to be deployed in the cluster as horizontal pod autoscaler uses it to collect metrics -(if you followed [getting started on GCE guide](../../../docs/getting-started-guides/gce.html), +(if you followed [getting started on GCE guide](/{{page.version}}/docs/getting-started-guides/gce), heapster monitoring will be turned-on by default). @@ -31,42 +27,42 @@ First, we will start a replication controller running the image and expose it as {% highlight console %} -{% raw %} + $ kubectl run php-apache --image=gcr.io/google_containers/hpa-example --requests=cpu=200m replicationcontroller "php-apache" created $ kubectl expose rc php-apache --port=80 --type=LoadBalancer service "php-apache" exposed -{% endraw %} + {% endhighlight %} Now, we will wait some time and verify that both the replication controller and the service were correctly created and are running. We will also determine the IP address of the service: {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY STATUS RESTARTS AGE php-apache-wa3t1 1/1 Running 0 12m $ kubectl describe services php-apache | grep "LoadBalancer Ingress" LoadBalancer Ingress: 146.148.24.244 -{% endraw %} + {% endhighlight %} We may now check that php-apache server works correctly by calling ``curl`` with the service's IP: {% highlight console %} -{% raw %} + $ curl http://146.148.24.244 OK! -{% endraw %} + {% endhighlight %} Please notice that when exposing the service we assumed that our cluster runs on a provider which supports load balancers (e.g.: on GCE). If load balancers are not supported (e.g.: on Vagrant), we can expose php-apache service as ``ClusterIP`` and connect to it using the proxy on the master: {% highlight console %} -{% raw %} + $ kubectl expose rc php-apache --port=80 --type=ClusterIP service "php-apache" exposed @@ -75,7 +71,7 @@ Kubernetes master is running at https://146.148.6.215 $ curl -k -u : https://146.148.6.215/api/v1/proxy/namespaces/default/services/php-apache/ OK! -{% endraw %} + {% endhighlight %} @@ -85,7 +81,7 @@ Now that the server is running, we will create a horizontal pod autoscaler for i To create it, we will use the [hpa-php-apache.yaml](hpa-php-apache.yaml) file, which looks like this: {% highlight yaml %} -{% raw %} + apiVersion: extensions/v1beta1 kind: HorizontalPodAutoscaler metadata: @@ -100,7 +96,7 @@ spec: maxReplicas: 10 cpuUtilization: targetPercentage: 50 -{% endraw %} + {% endhighlight %} This defines a horizontal pod autoscaler that maintains between 1 and 10 replicas of the Pods @@ -108,35 +104,35 @@ controlled by the php-apache replication controller we created in the first step Roughly speaking, the horizontal autoscaler will increase and decrease the number of replicas (via the replication controller) so as to maintain an average CPU utilization across all Pods of 50% (since each pod requests 200 milli-cores by [kubectl run](#kubectl-run), this means average CPU utilization of 100 milli-cores). -See [here](../../../docs/design/horizontal-pod-autoscaler.html#autoscaling-algorithm) for more details on the algorithm. +See [here](/{{page.version}}/docs/design/horizontal-pod-autoscaler.html#autoscaling-algorithm) for more details on the algorithm. We will create the autoscaler by executing the following command: {% highlight console %} -{% raw %} + $ kubectl create -f docs/user-guide/horizontal-pod-autoscaling/hpa-php-apache.yaml horizontalpodautoscaler "php-apache" created -{% endraw %} + {% endhighlight %} -Alternatively, we can create the autoscaler using [kubectl autoscale](../kubectl/kubectl_autoscale.html). +Alternatively, we can create the autoscaler using [kubectl autoscale](../kubectl/kubectl_autoscale). The following command will create the equivalent autoscaler as defined in the [hpa-php-apache.yaml](hpa-php-apache.yaml) file: ``` -{% raw %} + $ kubectl autoscale rc php-apache --cpu-percent=50 --min=1 --max=10 replicationcontroller "php-apache" autoscaled -{% endraw %} + ``` We may check the current status of autoscaler by running: {% highlight console %} -{% raw %} + $ kubectl get hpa NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE php-apache ReplicationController/default/php-apache/ 50% 0% 1 10 27s -{% endraw %} + {% endhighlight %} Please note that the current CPU consumption is 0% as we are not sending any requests to the server @@ -148,44 +144,44 @@ Now, we will see how the autoscaler reacts on the increased load of the server. We will start an infinite loop of queries to our server (please run it in a different terminal): {% highlight console %} -{% raw %} + $ while true; do curl http://146.148.6.244; done -{% endraw %} + {% endhighlight %} We may examine, how CPU load was increased (the results should be visible after about 3-4 minutes) by executing: {% highlight console %} -{% raw %} + $ kubectl get hpa NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE php-apache ReplicationController/default/php-apache/ 50% 305% 1 10 4m -{% endraw %} + {% endhighlight %} In the case presented here, it bumped CPU consumption to 305% of the request. As a result, the replication controller was resized to 7 replicas: {% highlight console %} -{% raw %} + $ kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE php-apache php-apache gcr.io/google_containers/hpa-example run=php-apache 7 18m -{% endraw %} + {% endhighlight %} Now, we may increase the load even more by running yet another infinite loop of queries (in yet another terminal): {% highlight console %} -{% raw %} + $ while true; do curl http://146.148.6.244; done -{% endraw %} + {% endhighlight %} In the case presented here, it increased the number of serving pods to 10: {% highlight console %} -{% raw %} + $ kubectl get hpa NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE php-apache ReplicationController/default/php-apache/ 50% 65% 1 10 14m @@ -193,7 +189,7 @@ php-apache ReplicationController/default/php-apache/ 50% 65% 1 $ kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE php-apache php-apache gcr.io/google_containers/hpa-example run=php-apache 10 24m -{% endraw %} + {% endhighlight %} ## Step Four: Stop load @@ -202,7 +198,7 @@ We will finish our example by stopping the user load. We will terminate both infinite ``while`` loops sending requests to the server and verify the result state: {% highlight console %} -{% raw %} + $ kubectl get hpa NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE php-apache ReplicationController/default/php-apache/ 50% 0% 1 10 21m @@ -210,7 +206,7 @@ php-apache ReplicationController/default/php-apache/ 50% 0% 1 $ kubectl get rc CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE php-apache php-apache gcr.io/google_containers/hpa-example run=php-apache 1 31m -{% endraw %} + {% endhighlight %} As we see, in the presented case CPU utilization dropped to 0, and the number of replicas dropped to 1. diff --git a/v1.1/docs/user-guide/identifiers.md b/v1.1/docs/user-guide/identifiers.md index ab6a40be2d..12c3d2ab76 100644 --- a/v1.1/docs/user-guide/identifiers.md +++ b/v1.1/docs/user-guide/identifiers.md @@ -1,17 +1,13 @@ --- title: "Identifiers" --- - - -# Identifiers - All objects in the Kubernetes REST API are unambiguously identified by a Name and a UID. -For non-unique user-provided attributes, Kubernetes provides [labels](labels.html) and [annotations](annotations.html). +For non-unique user-provided attributes, Kubernetes provides [labels](labels) and [annotations](annotations). ## Names -Names are generally client-provided. Only one object of a given kind can have a given name at a time (i.e., they are spatially unique). But if you delete an object, you can make a new object with the same name. Names are the used to refer to an object in a resource URL, such as `/api/v1/pods/some-name`. By convention, the names of Kubernetes resources should be up to maximum length of 253 characters and consist of lower case alphanumeric characters, `-`, and `.`, but certain resources have more specific restrictions. See the [identifiers design doc](../design/identifiers.html) for the precise syntax rules for names. +Names are generally client-provided. Only one object of a given kind can have a given name at a time (i.e., they are spatially unique). But if you delete an object, you can make a new object with the same name. Names are the used to refer to an object in a resource URL, such as `/api/v1/pods/some-name`. By convention, the names of Kubernetes resources should be up to maximum length of 253 characters and consist of lower case alphanumeric characters, `-`, and `.`, but certain resources have more specific restrictions. See the [identifiers design doc](../design/identifiers) for the precise syntax rules for names. ## UIDs diff --git a/v1.1/docs/user-guide/images.md b/v1.1/docs/user-guide/images.md index dd82818810..dfbf848cdf 100644 --- a/v1.1/docs/user-guide/images.md +++ b/v1.1/docs/user-guide/images.md @@ -1,29 +1,14 @@ --- title: "Images" --- - - -# Images - Each container in a pod has its own image. Currently, the only type of image supported is a [Docker Image](https://docs.docker.com/userguide/dockerimages/). You create your Docker image and push it to a registry before referring to it in a Kubernetes pod. The `image` property of a container supports the same syntax as the `docker` command does, including private registries and tags. -**Table of Contents** - +{% include pagetoc.html %} -- [Images](#images) - - [Updating Images](#updating-images) - - [Using a Private Registry](#using-a-private-registry) - - [Using Google Container Registry](#using-google-container-registry) - - [Configuring Nodes to Authenticate to a Private Repository](#configuring-nodes-to-authenticate-to-a-private-repository) - - [Pre-pulling Images](#pre-pulling-images) - - [Specifying ImagePullSecrets on a Pod](#specifying-imagepullsecrets-on-a-pod) - - [Use Cases](#use-cases) - - ## Updating Images @@ -89,7 +74,7 @@ example, run these on your desktop/laptop: Verify by creating a pod that uses a private image, e.g.: {% highlight yaml %} -{% raw %} + $ cat < /tmp/private-image-test-1.yaml apiVersion: v1 kind: Pod @@ -105,25 +90,25 @@ EOF $ kubectl create -f /tmp/private-image-test-1.yaml pods/private-image-test-1 $ -{% endraw %} + {% endhighlight %} If everything is working, then, after a few moments, you should see: {% highlight console %} -{% raw %} + $ kubectl logs private-image-test-1 SUCCESS -{% endraw %} + {% endhighlight %} If it failed, then you will see: {% highlight console %} -{% raw %} + $ kubectl describe pods/private-image-test-1 | grep "Failed" Fri, 26 Jun 2015 15:36:13 -0700 Fri, 26 Jun 2015 15:39:13 -0700 19 {kubelet node-i2hq} spec.containers{uses-private-image} failed Failed to pull image "user/privaterepo:v1": Error: image user/privaterepo:v1 not found -{% endraw %} + {% endhighlight %} @@ -165,13 +150,13 @@ where node creation is automated. Kubernetes supports specifying registry keys on a pod. First, create a `.dockercfg`, such as running `docker login `. -Then put the resulting `.dockercfg` file into a [secret resource](secrets.html). For example: +Then put the resulting `.dockercfg` file into a [secret resource](secrets). For example: {% highlight console %} -{% raw %} + $ docker login Username: janedoe -Password: ����������� +Password: '�?'�?'�?'�?'�?'�?'�?'�?'�?'�?'�? Email: jdoe@example.com WARNING: login credentials saved in /Users/jdoe/.dockercfg. Login Succeeded @@ -195,7 +180,7 @@ EOF $ kubectl create -f /tmp/image-pull-secret.yaml secrets/myregistrykey $ -{% endraw %} + {% endhighlight %} If you get the error message `error: no objects passed to create`, it may mean the base64 encoded string is invalid. @@ -208,7 +193,7 @@ Now, you can create pods which reference that secret by adding an `imagePullSecr section to a pod definition. {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: Pod metadata: @@ -219,12 +204,12 @@ spec: image: janedoe/awesomeapp:v1 imagePullSecrets: - name: myregistrykey -{% endraw %} + {% endhighlight %} This needs to be done for each pod that is using a private registry. However, setting of this field can be automated by setting the imagePullSecrets -in a [serviceAccount](service-accounts.html) resource. +in a [serviceAccount](service-accounts) resource. Currently, all pods will potentially have read access to any images which were pulled using imagePullSecrets. That is, imagePullSecrets does *NOT* protect your @@ -258,6 +243,3 @@ common use cases and suggested solutions. - DO NOT use imagePullSecrets for this use case yet. 1. A multi-tenant cluster where each tenant needs own private registry - NOT supported yet. - - - diff --git a/v1.1/docs/user-guide/index.md b/v1.1/docs/user-guide/index.md index 5aa731f8ee..d253184575 100644 --- a/v1.1/docs/user-guide/index.md +++ b/v1.1/docs/user-guide/index.md @@ -1,103 +1,90 @@ --- title: "Kubernetes User Guide: Managing Applications" --- +{% include pagetoc.html %} +The user guide is intended for anyone who wants to run programs and services on an existing Kubernetes cluster. Setup and administration of a Kubernetes cluster is described in the [Cluster Admin Guide](/{{page.version}}/docs/admin/README). The [Developer Guide](/{{page.version}}/docs/devel/README) is for anyone wanting to either write code which directly accesses the Kubernetes API, or to contribute directly to the Kubernetes project. -# Kubernetes User Guide: Managing Applications - -**Table of Contents** - - -- [Kubernetes User Guide: Managing Applications](#kubernetes-user-guide-managing-applications) - - [Quick walkthrough](#quick-walkthrough) - - [Thorough walkthrough](#thorough-walkthrough) - - [Concept guide](#concept-guide) - - [Further reading](#further-reading) - - - -The user guide is intended for anyone who wants to run programs and services on an existing Kubernetes cluster. Setup and administration of a Kubernetes cluster is described in the [Cluster Admin Guide](../../docs/admin/README.html). The [Developer Guide](../../docs/devel/README.html) is for anyone wanting to either write code which directly accesses the Kubernetes API, or to contribute directly to the Kubernetes project. - -Please ensure you have completed the [prerequisites for running examples from the user guide](prereqs.html). +Please ensure you have completed the [prerequisites for running examples from the user guide](prereqs). ## Quick walkthrough -1. [Kubernetes 101](walkthrough/README.html) -1. [Kubernetes 201](walkthrough/k8s201.html) +1. [Kubernetes 101](walkthrough/README) +1. [Kubernetes 201](walkthrough/k8s201) ## Thorough walkthrough If you don't have much familiarity with Kubernetes, we recommend you read the following sections in order: -1. [Quick start: launch and expose an application](quick-start.html) -1. [Configuring and launching containers: configuring common container parameters](configuring-containers.html) -1. [Deploying continuously running applications](deploying-applications.html) -1. [Connecting applications: exposing applications to clients and users](connecting-applications.html) -1. [Working with containers in production](production-pods.html) -1. [Managing deployments](managing-deployments.html) -1. [Application introspection and debugging](introspection-and-debugging.html) - 1. [Using the Kubernetes web user interface](ui.html) - 1. [Logging](logging.html) - 1. [Monitoring](monitoring.html) - 1. [Getting into containers via `exec`](getting-into-containers.html) - 1. [Connecting to containers via proxies](connecting-to-applications-proxy.html) - 1. [Connecting to containers via port forwarding](connecting-to-applications-port-forward.html) +1. [Quick start: launch and expose an application](quick-start) +1. [Configuring and launching containers: configuring common container parameters](configuring-containers) +1. [Deploying continuously running applications](deploying-applications) +1. [Connecting applications: exposing applications to clients and users](connecting-applications) +1. [Working with containers in production](production-pods) +1. [Managing deployments](managing-deployments) +1. [Application introspection and debugging](introspection-and-debugging) + 1. [Using the Kubernetes web user interface](ui) + 1. [Logging](logging) + 1. [Monitoring](monitoring) + 1. [Getting into containers via `exec`](getting-into-containers) + 1. [Connecting to containers via proxies](connecting-to-applications-proxy) + 1. [Connecting to containers via port forwarding](connecting-to-applications-port-forward) ## Concept guide -[**Overview**](overview.html) +[**Overview**](overview) : A brief overview of Kubernetes concepts. -[**Cluster**](../admin/README.html) +[**Cluster**](../admin/README) : A cluster is a set of physical or virtual machines and other infrastructure resources used by Kubernetes to run your applications. -[**Node**](../admin/node.html) +[**Node**](../admin/node) : A node is a physical or virtual machine running Kubernetes, onto which pods can be scheduled. -[**Pod**](pods.html) +[**Pod**](pods) : A pod is a co-located group of containers and volumes. -[**Label**](labels.html) +[**Label**](labels) : A label is a key/value pair that is attached to a resource, such as a pod, to convey a user-defined identifying attribute. Labels can be used to organize and to select subsets of resources. [**Selector**](labels.html#label-selectors) : A selector is an expression that matches labels in order to identify related resources, such as which pods are targeted by a load-balanced service. -[**Replication Controller**](replication-controller.html) +[**Replication Controller**](replication-controller) : A replication controller ensures that a specified number of pod replicas are running at any one time. It both allows for easy scaling of replicated systems and handles re-creation of a pod when the machine it is on reboots or otherwise fails. -[**Service**](services.html) +[**Service**](services) : A service defines a set of pods and a means by which to access them, such as single stable IP address and corresponding DNS name. -[**Volume**](volumes.html) +[**Volume**](volumes) : A volume is a directory, possibly with some data in it, which is accessible to a Container as part of its filesystem. Kubernetes volumes build upon [Docker Volumes](https://docs.docker.com/userguide/dockervolumes/), adding provisioning of the volume directory and/or device. -[**Secret**](secrets.html) +[**Secret**](secrets) : A secret stores sensitive data, such as authentication tokens, which can be made available to containers upon request. -[**Name**](identifiers.html) +[**Name**](identifiers) : A user- or client-provided name for a resource. -[**Namespace**](namespaces.html) +[**Namespace**](namespaces) : A namespace is like a prefix to the name of a resource. Namespaces help different projects, teams, or customers to share a cluster, such as by preventing name collisions between unrelated teams. -[**Annotation**](annotations.html) +[**Annotation**](annotations) : A key/value pair that can hold larger (compared to a label), and possibly not human-readable, data, intended to store non-identifying auxiliary data, especially data manipulated by tools and system extensions. Efficient filtering by annotation values is not supported. ## Further reading * API resources - * [Working with resources](working-with-resources.html) + * [Working with resources](working-with-resources) * Pods and containers - * [Pod lifecycle and restart policies](pod-states.html) - * [Lifecycle hooks](container-environment.html) - * [Compute resources, such as cpu and memory](compute-resources.html) - * [Specifying commands and requesting capabilities](containers.html) - * [Downward API: accessing system configuration from a pod](downward-api.html) - * [Images and registries](images.html) - * [Migrating from docker-cli to kubectl](docker-cli-to-kubectl.html) - * [Tips and tricks when working with config](config-best-practices.html) + * [Pod lifecycle and restart policies](pod-states) + * [Lifecycle hooks](container-environment) + * [Compute resources, such as cpu and memory](compute-resources) + * [Specifying commands and requesting capabilities](containers) + * [Downward API: accessing system configuration from a pod](downward-api) + * [Images and registries](images) + * [Migrating from docker-cli to kubectl](docker-cli-to-kubectl) + * [Tips and tricks when working with config](config-best-practices) * [Assign pods to selected nodes](node-selection/) * [Perform a rolling update on a running group of pods](update-demo/) diff --git a/v1.1/docs/user-guide/ingress.md b/v1.1/docs/user-guide/ingress.md index c6975b3c47..1fe02aaccb 100644 --- a/v1.1/docs/user-guide/ingress.md +++ b/v1.1/docs/user-guide/ingress.md @@ -1,28 +1,7 @@ --- title: "Ingress" --- - - -# Ingress - -**Table of Contents** - - -- [Ingress](#ingress) - - [What is Ingress?](#what-is-ingress) - - [Prerequisites](#prerequisites) - - [The Ingress Resource](#the-ingress-resource) - - [Ingress controllers](#ingress-controllers) - - [Types of Ingress](#types-of-ingress) - - [Single Service Ingress](#single-service-ingress) - - [Simple fanout](#simple-fanout) - - [Name based virtual hosting](#name-based-virtual-hosting) - - [Loadbalancing](#loadbalancing) - - [Updating an Ingress](#updating-an-ingress) - - [Future Work](#future-work) - - [Alternatives](#alternatives) - - +{% include pagetoc.html %} __Terminology__ @@ -39,24 +18,24 @@ Throughout this doc you will see a few terms that are sometimes used interchanga Typically, services and pods have IPs only routable by the cluster network. All traffic that ends up at an edge router is either dropped or forwarded elsewhere. Conceptually, this might look like: ``` -{% raw %} + internet | ------------ [ Services ] -{% endraw %} + ``` An Ingress is a collection of rules that allow inbound connections to reach the cluster services. ``` -{% raw %} + internet | [ Ingress ] --|-----|-- [ Services ] -{% endraw %} + ``` It can be configured to give services externally-reachable urls, load balance traffic, terminate SSL, offer name based virtual hosting etc. Users request ingress by POSTing the Ingress resource to the API server. An [Ingress controller](#ingress-controllers) is responsible for fulfilling the Ingress, usually with a loadbalancer, though it may also configure your edge router or additional frontends to help handle the traffic in an HA manner. @@ -74,7 +53,7 @@ Before you start using the Ingress resource, there are a few things you should u A minimal Ingress might look like: {% highlight yaml %} -{% raw %} + 01. apiVersion: extensions/v1beta1 02. kind: Ingress 03. metadata: @@ -87,18 +66,18 @@ A minimal Ingress might look like: 10. backend: 11. serviceName: test 12. servicePort: 80 -{% endraw %} + {% endhighlight %} *POSTing this to the API server will have no effect if you have not configured an [Ingress controller](#ingress-controllers).* -__Lines 1-4__: As with all other Kubernetes config, an Ingress needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [here](simple-yaml.html), [here](configuring-containers.html), and [here](working-with-resources.html). +__Lines 1-4__: As with all other Kubernetes config, an Ingress needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [here](simple-yaml), [here](configuring-containers), and [here](working-with-resources). __Lines 5-7__: Ingress [spec](../devel/api-conventions.html#spec-and-status) has all the information needed to configure a loadbalancer or proxy server. Most importantly, it contains a list of rules matched against all incoming requests. Currently the Ingress resource only supports http rules. __Lines 8-9__: Each http rule contains the following information: A host (eg: foo.bar.com, defaults to * in this example), a list of paths (eg: /testpath) each of which has an associated backend (test:80). Both the host and path must match the content of an incoming request before the loadbalancer directs traffic to the backend. -__Lines 10-12__: A backend is a service:port combination as described in the [services doc](services.html). Ingress traffic is typically sent directly to the endpoints matching a backend. +__Lines 10-12__: A backend is a service:port combination as described in the [services doc](services). Ingress traffic is typically sent directly to the endpoints matching a backend. __Global Parameters__: For the sake of simplicity the example Ingress has no global parameters, see the [api-reference](https://releases.k8s.io/release-1.1/pkg/apis/extensions/v1beta1/types.go) for a full definition of the resource. One can specify a global default backend in the absence of which requests that don't match a path in the spec are sent to the default backend of the Ingress controller. Though the Ingress resource doesn't support HTTPS yet, security configs would also be global. @@ -115,7 +94,7 @@ There are existing Kubernetes concepts that allow you to expose a single service {% highlight yaml %} -{% raw %} + apiVersion: extensions/v1beta1 kind: Ingress metadata: @@ -124,7 +103,7 @@ spec: backend: serviceName: testsvc servicePort: 80 -{% endraw %} + {% endhighlight %} [Download example](ingress.yaml) @@ -133,11 +112,11 @@ spec: If you create it using `kubectl -f` you should see: {% highlight sh %} -{% raw %} + $ kubectl get ing NAME RULE BACKEND ADDRESS test-ingress - testsvc:80 107.178.254.228 -{% endraw %} + {% endhighlight %} Where `107.178.254.228` is the IP allocated by the Ingress controller to satisfy this Ingress. The `RULE` column shows that all traffic send to the IP is directed to the Kubernetes Service listed under `BACKEND`. @@ -147,16 +126,16 @@ Where `107.178.254.228` is the IP allocated by the Ingress controller to satisfy As described previously, pods within kubernetes have ips only visible on the cluster network, so we need something at the edge accepting ingress traffic and proxying it to the right endpoints. This component is usually a highly available loadbalancer/s. An Ingress allows you to keep the number of loadbalancers down to a minimum, for example, a setup like: ``` -{% raw %} + foo.bar.com -> 178.91.123.132 -> / foo s1:80 / bar s2:80 -{% endraw %} + ``` would require an Ingress such as: {% highlight yaml %} -{% raw %} + apiVersion: extensions/v1beta1 kind: Ingress metadata: @@ -174,20 +153,20 @@ spec: backend: serviceName: s2 servicePort: 80 -{% endraw %} + {% endhighlight %} When you create the Ingress with `kubectl create -f`: ``` -{% raw %} + $ kubectl get ing NAME RULE BACKEND ADDRESS test - foo.bar.com /foo s1:80 /bar s2:80 -{% endraw %} + ``` The Ingress controller will provision an implementation specific loadbalancer that satisfies the Ingress, as long as the services (s1, s2) exist. When it has done so, you will see the address of the loadbalancer under the last column of the Ingress. @@ -197,18 +176,18 @@ The Ingress controller will provision an implementation specific loadbalancer th Name-based virtual hosts use multiple host names for the same IP address. ``` -{% raw %} + foo.bar.com --| |-> foo.bar.com s1:80 | 178.91.123.132 | bar.foo.com --| |-> bar.foo.com s2:80 -{% endraw %} + ``` The following Ingress tells the backing loadbalancer to route requests based on the [Host header](https://tools.ietf.org/html/rfc7230#section-5.4). {% highlight yaml %} -{% raw %} + apiVersion: extensions/v1beta1 kind: Ingress metadata: @@ -227,7 +206,7 @@ spec: - backend: serviceName: s2 servicePort: 80 -{% endraw %} + {% endhighlight %} @@ -244,20 +223,20 @@ It's also worth noting that even though health checks are not exposed directly t Say you'd like to add a new Host to an existing Ingress, you can update it by editing the resource: {% highlight sh %} -{% raw %} + $ kubectl get ing NAME RULE BACKEND ADDRESS test - 178.91.123.132 foo.bar.com /foo s1:80 $ kubectl edit ing test -{% endraw %} + {% endhighlight %} This should pop up an editor with the existing yaml, modify it to include the new Host. {% highlight yaml %} -{% raw %} + spec: rules: - host: foo.bar.com @@ -275,13 +254,13 @@ spec: servicePort: 80 path: /foo .. -{% endraw %} + {% endhighlight %} saving it will update the resource in the API server, which should tell the Ingress controller to reconfigure the loadbalancer. {% highlight sh %} -{% raw %} + $ kubectl get ing NAME RULE BACKEND ADDRESS test - 178.91.123.132 @@ -289,7 +268,7 @@ test - 178.91.123.132 /foo s1:80 bar.baz.com /foo s2:80 -{% endraw %} + {% endhighlight %} You can achieve the same by invoking `kubectl replace -f` on a modified Ingress yaml file. diff --git a/v1.1/docs/user-guide/introspection-and-debugging.md b/v1.1/docs/user-guide/introspection-and-debugging.md index 6769923f09..b816ea9b2f 100644 --- a/v1.1/docs/user-guide/introspection-and-debugging.md +++ b/v1.1/docs/user-guide/introspection-and-debugging.md @@ -1,31 +1,18 @@ --- title: "Kubernetes User Guide: Managing Applications: Application Introspection and Debugging" --- - - -# Kubernetes User Guide: Managing Applications: Application Introspection and Debugging - -Once your application is running, you’ll inevitably need to debug problems with it. +Once your application is running, you'll inevitably need to debug problems with it. Earlier we described how you can use `kubectl get pods` to retrieve simple status information about your pods. But there are a number of ways to get even more information about your application. -**Table of Contents** - - -- [Kubernetes User Guide: Managing Applications: Application Introspection and Debugging](#kubernetes-user-guide-managing-applications-application-introspection-and-debugging) - - [Using `kubectl describe pod` to fetch details about pods](#using-kubectl-describe-pod-to-fetch-details-about-pods) - - [Example: debugging Pending Pods](#example-debugging-pending-pods) - - [Example: debugging a down/unreachable node](#example-debugging-a-downunreachable-node) - - [What's next?](#whats-next) - - +{% include pagetoc.html %} ## Using `kubectl describe pod` to fetch details about pods -For this example we’ll use a ReplicationController to create two pods, similar to the earlier example. +For this example we'll use a ReplicationController to create two pods, similar to the earlier example. {% highlight yaml %} -{% raw %} + apiVersion: v1 kind: ReplicationController metadata: @@ -46,29 +33,29 @@ spec: cpu: "500m" ports: - containerPort: 80 -{% endraw %} + {% endhighlight %} {% highlight console %} -{% raw %} + $ kubectl create -f ./my-nginx-rc.yaml replicationcontrollers/my-nginx -{% endraw %} + {% endhighlight %} {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY REASON RESTARTS AGE my-nginx-gy1ij 1/1 Running 0 1m my-nginx-yv5cn 1/1 Running 0 1m -{% endraw %} + {% endhighlight %} We can retrieve a lot more information about each of these pods using `kubectl describe pod`. For example: {% highlight console %} -{% raw %} + $ kubectl describe pod my-nginx-gy1ij Name: my-nginx-gy1ij Image(s): nginx @@ -101,7 +88,7 @@ Events: Thu, 09 Jul 2015 15:33:06 -0700 Thu, 09 Jul 2015 15:33:06 -0700 1 {kubelet kubernetes-minion-y3vk} spec.containers{nginx} pulled Successfully pulled image "nginx" Thu, 09 Jul 2015 15:33:06 -0700 Thu, 09 Jul 2015 15:33:06 -0700 1 {kubelet kubernetes-minion-y3vk} spec.containers{nginx} created Created with docker id 56d7a7b14dac Thu, 09 Jul 2015 15:33:07 -0700 Thu, 09 Jul 2015 15:33:07 -0700 1 {kubelet kubernetes-minion-y3vk} spec.containers{nginx} started Started with docker id 56d7a7b14dac -{% endraw %} + {% endhighlight %} Here you can see configuration information about the container(s) and Pod (labels, resource requirements, etc.), as well as status information about the container(s) and Pod (state, readiness, restart count, events, etc.) @@ -110,7 +97,7 @@ The container state is one of Waiting, Running, or Terminated. Depending on the Ready tells you whether the container passed its last readiness probe. (In this case, the container does not have a readiness probe configured; the container is assumed to be ready if no readiness probe is configured.) -Restart Count tells you how many times the container has restarted; this information can be useful for detecting crash loops in containers that are configured with a restart policy of “always.� +Restart Count tells you how many times the container has restarted; this information can be useful for detecting crash loops in containers that are configured with a restart policy of 'always.'? Currently the only Condition associated with a Pod is the binary Ready condition, which indicates that the pod is able to service requests and should be added to the load balancing pools of all matching services. @@ -118,10 +105,10 @@ Lastly, you see a log of recent events related to your Pod. The system compresse ## Example: debugging Pending Pods -A common scenario that you can detect using events is when you’ve created a Pod that won’t fit on any node. For example, the Pod might request more resources than are free on any node, or it might specify a label selector that doesn’t match any nodes. Let’s say we created the previous Replication Controller with 5 replicas (instead of 2) and requesting 600 millicores instead of 500, on a four-node cluster where each (virtual) machine has 1 CPU. In that case one of the Pods will not be able to schedule. (Note that because of the cluster addon pods such as fluentd, skydns, etc., that run on each node, if we requested 1000 millicores then none of the Pods would be able to schedule.) +A common scenario that you can detect using events is when you've created a Pod that won't fit on any node. For example, the Pod might request more resources than are free on any node, or it might specify a label selector that doesn't match any nodes. Let's say we created the previous Replication Controller with 5 replicas (instead of 2) and requesting 600 millicores instead of 500, on a four-node cluster where each (virtual) machine has 1 CPU. In that case one of the Pods will not be able to schedule. (Note that because of the cluster addon pods such as fluentd, skydns, etc., that run on each node, if we requested 1000 millicores then none of the Pods would be able to schedule.) {% highlight console %} -{% raw %} + $ kubectl get pods NAME READY REASON RESTARTS AGE my-nginx-9unp9 0/1 Pending 0 8s @@ -129,13 +116,13 @@ my-nginx-b7zs9 0/1 Running 0 8s my-nginx-i595c 0/1 Running 0 8s my-nginx-iichp 0/1 Running 0 8s my-nginx-tc2j9 0/1 Running 0 8s -{% endraw %} + {% endhighlight %} To find out why the my-nginx-9unp9 pod is not running, we can use `kubectl describe pod` on the pending Pod and look at its events: {% highlight console %} -{% raw %} + $ kubectl describe pod my-nginx-9unp9 Name: my-nginx-9unp9 Image(s): nginx @@ -158,7 +145,7 @@ Containers: Events: FirstSeen LastSeen Count From SubobjectPath Reason Message Thu, 09 Jul 2015 23:56:21 -0700 Fri, 10 Jul 2015 00:01:30 -0700 21 {scheduler } failedScheduling Failed for reason PodFitsResources and possibly others -{% endraw %} + {% endhighlight %} Here you can see the event generated by the scheduler saying that the Pod failed to schedule for reason `PodFitsResources` (and possibly others). `PodFitsResources` means there were not enough resources for the Pod on any of the nodes. Due to the way the event is generated, there may be other reasons as well, hence "and possibly others." @@ -168,17 +155,17 @@ To correct this situation, you can use `kubectl scale` to update your Replicatio Events such as the ones you saw at the end of `kubectl describe pod` are persisted in etcd and provide high-level information on what is happening in the cluster. To list all events you can use ``` -{% raw %} + kubectl get events -{% endraw %} + ``` but you have to remember that events are namespaced. This means that if you're interested in events for some namespaced object (e.g. what happened with Pods in namespace `my-namespace`) you need to explicitly provide a namespace to the command: ``` -{% raw %} + kubectl get events --namespace=my-namespace -{% endraw %} + ``` To see events from all namespaces, you can use the `--all-namespaces` argument. @@ -186,7 +173,7 @@ To see events from all namespaces, you can use the `--all-namespaces` argument. In addition to `kubectl describe pod`, another way to get extra information about a pod (beyond what is provided by `kubectl get pod`) is to pass the `-o yaml` output format flag to `kubectl get pod`. This will give you, in YAML format, even more information than `kubectl describe pod`--essentially all of the information the system has about the Pod. Here you will see things like annotations (which are key-value metadata without the label restrictions, that is used internally by Kubernetes system components), restart policy, ports, and volumes. {% highlight yaml %} -{% raw %} + $ kubectl get pod my-nginx-i595c -o yaml apiVersion: v1 kind: Pod @@ -246,15 +233,15 @@ status: phase: Running podIP: 10.244.3.4 startTime: 2015-07-10T06:56:21Z -{% endraw %} + {% endhighlight %} ## Example: debugging a down/unreachable node -Sometimes when debugging it can be useful to look at the status of a node -- for example, because you've noticed strange behavior of a Pod that’s running on the node, or to find out why a Pod won’t schedule onto the node. As with Pods, you can use `kubectl describe node` and `kubectl get node -o yaml` to retrieve detailed information about nodes. For example, here's what you'll see if a node is down (disconnected from the network, or kubelet dies and won't restart, etc.). Notice the events that show the node is NotReady, and also notice that the pods are no longer running (they are evicted after five minutes of NotReady status). +Sometimes when debugging it can be useful to look at the status of a node -- for example, because you've noticed strange behavior of a Pod that's running on the node, or to find out why a Pod won't schedule onto the node. As with Pods, you can use `kubectl describe node` and `kubectl get node -o yaml` to retrieve detailed information about nodes. For example, here's what you'll see if a node is down (disconnected from the network, or kubelet dies and won't restart, etc.). Notice the events that show the node is NotReady, and also notice that the pods are no longer running (they are evicted after five minutes of NotReady status). {% highlight console %} -{% raw %} + $ kubectl get nodes NAME LABELS STATUS kubernetes-minion-861h kubernetes.io/hostname=kubernetes-minion-861h NotReady @@ -333,17 +320,17 @@ status: machineID: "" osImage: Debian GNU/Linux 7 (wheezy) systemUUID: ABE5F6B4-D44B-108B-C46A-24CCE16C8B6E -{% endraw %} + {% endhighlight %} ## What's next? Learn about additional debugging tools, including: -* [Logging](logging.html) -* [Monitoring](monitoring.html) -* [Getting into containers via `exec`](getting-into-containers.html) -* [Connecting to containers via proxies](connecting-to-applications-proxy.html) -* [Connecting to containers via port forwarding](connecting-to-applications-port-forward.html) +* [Logging](logging) +* [Monitoring](monitoring) +* [Getting into containers via `exec`](getting-into-containers) +* [Connecting to containers via proxies](connecting-to-applications-proxy) +* [Connecting to containers via port forwarding](connecting-to-applications-port-forward) diff --git a/v1.1/docs/user-guide/jobs.md b/v1.1/docs/user-guide/jobs.md index e38fca2f4f..6af9e485c0 100644 --- a/v1.1/docs/user-guide/jobs.md +++ b/v1.1/docs/user-guide/jobs.md @@ -1,29 +1,7 @@ --- title: "Jobs" --- - - -# Jobs - -**Table of Contents** - - -- [Jobs](#jobs) - - [What is a _job_?](#what-is-a-job) - - [Running an example Job](#running-an-example-job) - - [Writing a Job Spec](#writing-a-job-spec) - - [Pod Template](#pod-template) - - [Pod Selector](#pod-selector) - - [Multiple Completions](#multiple-completions) - - [Parallelism](#parallelism) - - [Handling Pod and Container Failures](#handling-pod-and-container-failures) - - [Alternatives to Job](#alternatives-to-job) - - [Bare Pods](#bare-pods) - - [Replication Controller](#replication-controller) - - [Caveats](#caveats) - - [Future work](#future-work) - - +{% include pagetoc.html %} ## What is a _job_? @@ -42,7 +20,7 @@ It takes around 10s to complete. {% highlight yaml %} -{% raw %} + apiVersion: extensions/v1beta1 kind: Job metadata: @@ -62,7 +40,7 @@ spec: image: perl command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"] restartPolicy: Never -{% endraw %} + {% endhighlight %} [Download example](job.yaml) @@ -71,16 +49,16 @@ spec: Run the example job by downloading the example file and then running this command: {% highlight console %} -{% raw %} + $ kubectl create -f ./job.yaml jobs/pi -{% endraw %} + {% endhighlight %} Check on the status of the job using this command: {% highlight console %} -{% raw %} + $ kubectl describe jobs/pi Name: pi Namespace: default @@ -92,10 +70,10 @@ Labels: Pods Statuses: 1 Running / 0 Succeeded / 0 Failed Events: FirstSeen LastSeen Count From SubobjectPath Reason Message - ───────── ──────── ───── ──── ───────────── ────── ─────── + '��'��'��'��'��'��'��'��'�� '��'��'��'��'��'��'��'�� '��'��'��'��'�� '��'��'��'�� '��'��'��'��'��'��'��'��'��'��'��'��'�� '��'��'��'��'��'�� '��'��'��'��'��'��'�� 1m 1m 1 {job } SuccessfulCreate Created pod: pi-z548a -{% endraw %} + {% endhighlight %} To view completed pods of a job, use `kubectl get pods --show-all`. The `--show-all` will show completed pods too. @@ -103,11 +81,11 @@ To view completed pods of a job, use `kubectl get pods --show-all`. The `--show To list all the pods that belong to job in a machine readable form, you can use a command like this: {% highlight console %} -{% raw %} + $ pods=$(kubectl get pods --selector=app=pi --output=jsonpath={.items..metadata.name}) echo $pods pi-aiw0a -{% endraw %} + {% endhighlight %} Here, the selector is the same as the selector for the job. The `--output=jsonpath` option specifies an expression @@ -116,17 +94,17 @@ that just gets the name from each pod in the returned list. View the standard output of one of the pods: {% highlight console %} -{% raw %} + $ kubectl logs pi-aiw0a 3.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066063155881748815209209628292540917153643678925903600113305305488204665213841469519415116094330572703657595919530921861173819326117931051185480744623799627495673518857527248912279381830119491298336733624406566430860213949463952247371907021798609437027705392171762931767523846748184676694051320005681271452635608277857713427577896091736371787214684409012249534301465495853710507922796892589235420199561121290219608640344181598136297747713099605187072113499999983729780499510597317328160963185950244594553469083026425223082533446850352619311881710100031378387528865875332083814206171776691473035982534904287554687311595628638823537875937519577818577805321712268066130019278766111959092164201989380952572010654858632788659361533818279682303019520353018529689957736225994138912497217752834791315155748572424541506959508295331168617278558890750983817546374649393192550604009277016711390098488240128583616035637076601047101819429555961989467678374494482553797747268471040475346462080466842590694912933136770289891521047521620569660240580381501935112533824300355876402474964732639141992726042699227967823547816360093417216412199245863150302861829745557067498385054945885869269956909272107975093029553211653449872027559602364806654991198818347977535663698074265425278625518184175746728909777727938000816470600161452491921732172147723501414419735685481613611573525521334757418494684385233239073941433345477624168625189835694855620992192221842725502542568876717904946016534668049886272327917860857843838279679766814541009538837863609506800642251252051173929848960841284886269456042419652850222106611863067442786220391949450471237137869609563643719172874677646575739624138908658326459958133904780275901 -{% endraw %} + {% endhighlight %} ## Writing a Job Spec As with all other Kubernetes config, a Job needs `apiVersion`, `kind`, and `metadata` fields. For -general information about working with config files, see [here](simple-yaml.html), -[here](configuring-containers.html), and [here](working-with-resources.html). +general information about working with config files, see [here](simple-yaml), +[here](configuring-containers), and [here](working-with-resources). A Job also needs a [`.spec` section](../devel/api-conventions.html#spec-and-status). @@ -135,20 +113,20 @@ A Job also needs a [`.spec` section](../devel/api-conventions.html#spec-and-stat The `.spec.template` is the only required field of the `.spec`. The `.spec.template` is a [pod template](replication-controller.html#pod-template). It has exactly -the same schema as a [pod](pods.html), except it is nested and does not have an `apiVersion` or +the same schema as a [pod](pods), except it is nested and does not have an `apiVersion` or `kind`. In addition to required fields for a Pod, a pod template in a job must specify appropriate lables (see [pod selector](#pod-selector) and an appropriate restart policy. -Only a [`RestartPolicy`](pod-states.html) equal to `Never` or `OnFailure` are allowed. +Only a [`RestartPolicy`](pod-states) equal to `Never` or `OnFailure` are allowed. ### Pod Selector The `.spec.selector` field is a label query over a set of pods. The `spec.selector` is an object consisting of two fields: -* `matchLabels` - works the same as the `.spec.selector` of a [ReplicationController](replication-controller.html) +* `matchLabels` - works the same as the `.spec.selector` of a [ReplicationController](replication-controller) * `matchExpressions` - allows to build more sophisticated selectors by specyfing key, list of values and an operator that relates the key and values. @@ -192,7 +170,7 @@ a non-zero exit code, or the Container was killed for exceeding a memory limit, happens, and the `.spec.template.containers[].restartPolicy = "OnFailure"`, then the Pod stays on the node, but the Container is re-run. Therefore, your program needs to handle the the case when it is restarted locally, or else specify `.spec.template.containers[].restartPolicy = "Never"`. -See [pods-states](pod-states.html) for more information on `restartPolicy`. +See [pods-states](pod-states) for more information on `restartPolicy`. An entire Pod can also fail, for a number of reasons, such as when the pod is kicked off the node (node is upgraded, rebooted, delelted, etc.), or if a container of the Pod fails and the @@ -219,11 +197,11 @@ requires only a single pod. ### Replication Controller -Jobs are complementary to [Replication Controllers](replication-controller.html). +Jobs are complementary to [Replication Controllers](replication-controller). A Replication Controller manages pods which are not expected to terminate (e.g. web servers), and a Job manages pods that are expected to terminate (e.g. batch jobs). -As discussed in [life of a pod](pod-states.html), `Job` is *only* appropriate for pods with +As discussed in [life of a pod](pod-states), `Job` is *only* appropriate for pods with `RestartPolicy` equal to `OnFailure` or `Never`. (Note: If `RestartPolicy` is not set, the default value is `Always`.) @@ -239,5 +217,3 @@ similar functionality will be supported. Support for creating Jobs at specified times/dates (i.e. cron) is expected in the next minor release. - - diff --git a/v1.1/docs/user-guide/jsonpath.md b/v1.1/docs/user-guide/jsonpath.md index ce5d38f688..e1478a07ed 100644 --- a/v1.1/docs/user-guide/jsonpath.md +++ b/v1.1/docs/user-guide/jsonpath.md @@ -1,10 +1,6 @@ --- title: "JSONPath template syntax" --- - - -# JSONPath template syntax - JSONPath template is composed of JSONPath expressions enclosed by {}. And we add three functions in addition to the original JSONPath syntax: @@ -18,7 +14,7 @@ The result object is printed as its String() function. Given the input: {% highlight json %} -{% raw %} + { "kind": "List", "items":[ @@ -53,7 +49,7 @@ Given the input: } ] } -{% endraw %} + {% endhighlight %} Function | Description | Example | Result diff --git a/v1.1/docs/user-guide/known-issues.md b/v1.1/docs/user-guide/known-issues.md index 7863c65e3d..3f34ca9c90 100644 --- a/v1.1/docs/user-guide/known-issues.md +++ b/v1.1/docs/user-guide/known-issues.md @@ -1,10 +1,6 @@ --- title: "Known Issues" --- - - -## Known Issues - This document summarizes known issues with existing Kubernetes releases. Please consult this document before filing new bugs. diff --git a/v1.1/docs/user-guide/kubeconfig-file.md b/v1.1/docs/user-guide/kubeconfig-file.md index bba9d311cf..1d55e9e520 100644 --- a/v1.1/docs/user-guide/kubeconfig-file.md +++ b/v1.1/docs/user-guide/kubeconfig-file.md @@ -1,226 +1,213 @@ ---- -title: "Authenticating Across Clusters with kubeconfig" -section: guides ---- -Authentication in kubernetes can differ for different individuals. - -- A running kubelet might have one way of authenticating (i.e. certificates). -- Users might have a different way of authenticating (i.e. tokens). -- Administrators might have a list of certificates which they provide individual users. -- There may be multiple clusters, and we may want to define them all in one place - giving users the ability to use their own certificates and reusing the same global configuration. - -So in order to easily switch between multiple clusters, for multiple users, a kubeconfig file was defined. - -This file contains a series of authentication mechanisms and cluster connection information associated with nicknames. It also introduces the concept of a tuple of authentication information (user) and cluster connection information called a context that is also associated with a nickname. - -Multiple kubeconfig files are allowed. At runtime they are loaded and merged together along with override options specified from the command line (see rules below). - -## Related discussion - -http://issue.k8s.io/1755 - -## Example kubeconfig file - -The below file contains a `current-context` which will be used by default by clients which are using the file to connect to a cluster. Thus, this kubeconfig file has more information in it then we will necessarily have to use in a given session. You can see it defines many clusters, and users associated with those clusters. The context itself is associated with both a cluster AND a user. - -{% highlight yaml %} -{% raw %} -current-context: federal-context -apiVersion: v1 -clusters: -- cluster: - api-version: v1 - server: http://cow.org:8080 - name: cow-cluster -- cluster: - certificate-authority: path/to/my/cafile - server: https://horse.org:4443 - name: horse-cluster -- cluster: - insecure-skip-tls-verify: true - server: https://pig.org:443 - name: pig-cluster -contexts: -- context: - cluster: horse-cluster - namespace: chisel-ns - user: green-user - name: federal-context -- context: - cluster: pig-cluster - namespace: saw-ns - user: black-user - name: queen-anne-context -kind: Config -preferences: - colors: true -users: -- name: blue-user - user: - token: blue-token -- name: green-user - user: - client-certificate: path/to/my/client/cert - client-key: path/to/my/client/key -{% endraw %} -{% endhighlight %} - -### Building your own kubeconfig file - -NOTE, that if you are deploying k8s via kube-up.sh, you do not need to create your own kubeconfig files, the script will do it for you. - -In any case, you can easily use this file as a template to create your own kubeconfig files. - -So, lets do a quick walk through the basics of the above file so you can easily modify it as needed... - -The above file would likely correspond to an api-server which was launched using the `--token-auth-file=tokens.csv` option, where the tokens.csv file looked something like this: - -``` -{% raw %} -blue-user,blue-user,1 -mister-red,mister-red,2 -{% endraw %} -``` - -Also, since we have other users who validate using **other** mechanisms, the api-server would have probably been launched with other authentication options (there are many such options, make sure you understand which ones YOU care about before crafting a kubeconfig file, as nobody needs to implement all the different permutations of possible authentication schemes). - -- Since the user for the current context is "green-user", any client of the api-server using this kubeconfig file would naturally be able to log in succesfully, because we are providigin the green-user's client credentials. -- Similarly, we can operate as the "blue-user" if we choose to change the value of current-context. - -In the above scenario, green-user would have to log in by providing certificates, whereas blue-user would just provide the token. All this information would be handled for us by the - -## Loading and merging rules - -The rules for loading and merging the kubeconfig files are straightforward, but there are a lot of them. The final config is built in this order: - 1. Get the kubeconfig from disk. This is done with the following hierarchy and merge rules: - - - If the CommandLineLocation (the value of the `kubeconfig` command line option) is set, use this file only. No merging. Only one instance of this flag is allowed. - - - Else, if EnvVarLocation (the value of $KUBECONFIG) is available, use it as a list of files that should be merged. - Merge files together based on the following rules. - Empty filenames are ignored. Files with non-deserializable content produced errors. - The first file to set a particular value or map key wins and the value or map key is never changed. - This means that the first file to set CurrentContext will have its context preserved. It also means that if two files specify a "red-user", only values from the first file's red-user are used. Even non-conflicting entries from the second file's "red-user" are discarded. - - - Otherwise, use HomeDirectoryLocation (~/.kube/config) with no merging. - 1. Determine the context to use based on the first hit in this chain - 1. command line argument - the value of the `context` command line option - 1. current-context from the merged kubeconfig file - 1. Empty is allowed at this stage - 1. Determine the cluster info and user to use. At this point, we may or may not have a context. They are built based on the first hit in this chain. (run it twice, once for user, once for cluster) - 1. command line argument - `user` for user name and `cluster` for cluster name - 1. If context is present, then use the context's value - 1. Empty is allowed - 1. Determine the actual cluster info to use. At this point, we may or may not have a cluster info. Build each piece of the cluster info based on the chain (first hit wins): - 1. command line arguments - `server`, `api-version`, `certificate-authority`, and `insecure-skip-tls-verify` - 1. If cluster info is present and a value for the attribute is present, use it. - 1. If you don't have a server location, error. - 1. Determine the actual user info to use. User is built using the same rules as cluster info, EXCEPT that you can only have one authentication technique per user. - 1. Load precedence is 1) command line flag, 2) user fields from kubeconfig - 1. The command line flags are: `client-certificate`, `client-key`, `username`, `password`, and `token`. - 1. If there are two conflicting techniques, fail. - 1. For any information still missing, use default values and potentially prompt for authentication information - -## Manipulation of kubeconfig via `kubectl config ` - -In order to more easily manipulate kubeconfig files, there are a series of subcommands to `kubectl config` to help. -See [kubectl/kubectl_config.md](kubectl/kubectl_config.html) for help. - -### Example - -{% highlight console %} -{% raw %} -$ kubectl config set-credentials myself --username=admin --password=secret -$ kubectl config set-cluster local-server --server=http://localhost:8080 -$ kubectl config set-context default-context --cluster=local-server --user=myself -$ kubectl config use-context default-context -$ kubectl config set contexts.default-context.namespace the-right-prefix -$ kubectl config view -{% endraw %} -{% endhighlight %} - -produces this output - -{% highlight yaml %} -{% raw %} -apiVersion: v1 -clusters: -- cluster: - server: http://localhost:8080 - name: local-server -contexts: -- context: - cluster: local-server - namespace: the-right-prefix - user: myself - name: default-context -current-context: default-context -kind: Config -preferences: {} -users: -- name: myself - user: - password: secret - username: admin -{% endraw %} -{% endhighlight %} - -and a kubeconfig file that looks like this - -{% highlight yaml %} -{% raw %} -apiVersion: v1 -clusters: -- cluster: - server: http://localhost:8080 - name: local-server -contexts: -- context: - cluster: local-server - namespace: the-right-prefix - user: myself - name: default-context -current-context: default-context -kind: Config -preferences: {} -users: -- name: myself - user: - password: secret - username: admin -{% endraw %} -{% endhighlight %} - -#### Commands for the example file - -{% highlight console %} -{% raw %} -$ kubectl config set preferences.colors true -$ kubectl config set-cluster cow-cluster --server=http://cow.org:8080 --api-version=v1 -$ kubectl config set-cluster horse-cluster --server=https://horse.org:4443 --certificate-authority=path/to/my/cafile -$ kubectl config set-cluster pig-cluster --server=https://pig.org:443 --insecure-skip-tls-verify=true -$ kubectl config set-credentials blue-user --token=blue-token -$ kubectl config set-credentials green-user --client-certificate=path/to/my/client/cert --client-key=path/to/my/client/key -$ kubectl config set-context queen-anne-context --cluster=pig-cluster --user=black-user --namespace=saw-ns -$ kubectl config set-context federal-context --cluster=horse-cluster --user=green-user --namespace=chisel-ns -$ kubectl config use-context federal-context -{% endraw %} -{% endhighlight %} - -### Final notes for tying it all together - -So, tying this all together, a quick start to creating your own kubeconfig file: - -- Take a good look and understand how you're api-server is being launched: You need to know YOUR security requirements and policies before you can design a kubeconfig file for convenient authentication. - -- Replace the snippet above with information for your cluster's api-server endpoint. - -- Make sure your api-server is launched in such a way that at least one user (i.e. green-user) credentials are provided to it. You will of course have to look at api-server documentation in order to determine the current state-of-the-art in terms of providing authentication details. - - - - - +--- +title: "Authenticating Across Clusters with kubeconfig" +--- +Authentication in kubernetes can differ for different individuals. + +- A running kubelet might have one way of authenticating (i.e. certificates). +- Users might have a different way of authenticating (i.e. tokens). +- Administrators might have a list of certificates which they provide individual users. +- There may be multiple clusters, and we may want to define them all in one place - giving users the ability to use their own certificates and reusing the same global configuration. + +So in order to easily switch between multiple clusters, for multiple users, a kubeconfig file was defined. + +This file contains a series of authentication mechanisms and cluster connection information associated with nicknames. It also introduces the concept of a tuple of authentication information (user) and cluster connection information called a context that is also associated with a nickname. + +Multiple kubeconfig files are allowed. At runtime they are loaded and merged together along with override options specified from the command line (see rules below). + +## Related discussion + +http://issue.k8s.io/1755 + +## Example kubeconfig file + +The below file contains a `current-context` which will be used by default by clients which are using the file to connect to a cluster. Thus, this kubeconfig file has more information in it then we will necessarily have to use in a given session. You can see it defines many clusters, and users associated with those clusters. The context itself is associated with both a cluster AND a user. + +{% highlight yaml %} +current-context: federal-context +apiVersion: v1 +clusters: +- cluster: + api-version: v1 + server: http://cow.org:8080 + name: cow-cluster +- cluster: + certificate-authority: path/to/my/cafile + server: https://horse.org:4443 + name: horse-cluster +- cluster: + insecure-skip-tls-verify: true + server: https://pig.org:443 + name: pig-cluster +contexts: +- context: + cluster: horse-cluster + namespace: chisel-ns + user: green-user + name: federal-context +- context: + cluster: pig-cluster + namespace: saw-ns + user: black-user + name: queen-anne-context +kind: Config +preferences: + colors: true +users: +- name: blue-user + user: + token: blue-token +- name: green-user + user: + client-certificate: path/to/my/client/cert + client-key: path/to/my/client/key +{% endhighlight %} + +### Building your own kubeconfig file + +NOTE, that if you are deploying k8s via kube-up.sh, you do not need to create your own kubeconfig files, the script will do it for you. + +In any case, you can easily use this file as a template to create your own kubeconfig files. + +So, lets do a quick walk through the basics of the above file so you can easily modify it as needed... + +The above file would likely correspond to an api-server which was launched using the `--token-auth-file=tokens.csv` option, where the tokens.csv file looked something like this: + +``` +blue-user,blue-user,1 +mister-red,mister-red,2 +``` + +Also, since we have other users who validate using **other** mechanisms, the api-server would have probably been launched with other authentication options (there are many such options, make sure you understand which ones YOU care about before crafting a kubeconfig file, as nobody needs to implement all the different permutations of possible authentication schemes). + +- Since the user for the current context is "green-user", any client of the api-server using this kubeconfig file would naturally be able to log in succesfully, because we are providigin the green-user's client credentials. +- Similarly, we can operate as the "blue-user" if we choose to change the value of current-context. + +In the above scenario, green-user would have to log in by providing certificates, whereas blue-user would just provide the token. All this information would be handled for us by the + +## Loading and merging rules + +The rules for loading and merging the kubeconfig files are straightforward, but there are a lot of them. The final config is built in this order: + 1. Get the kubeconfig from disk. This is done with the following hierarchy and merge rules: + + + If the CommandLineLocation (the value of the `kubeconfig` command line option) is set, use this file only. No merging. Only one instance of this flag is allowed. + + + Else, if EnvVarLocation (the value of $KUBECONFIG) is available, use it as a list of files that should be merged. + Merge files together based on the following rules. + Empty filenames are ignored. Files with non-deserializable content produced errors. + The first file to set a particular value or map key wins and the value or map key is never changed. + This means that the first file to set CurrentContext will have its context preserved. It also means that if two files specify a "red-user", only values from the first file's red-user are used. Even non-conflicting entries from the second file's "red-user" are discarded. + + + Otherwise, use HomeDirectoryLocation (~/.kube/config) with no merging. + 1. Determine the context to use based on the first hit in this chain + 1. command line argument - the value of the `context` command line option + 1. current-context from the merged kubeconfig file + 1. Empty is allowed at this stage + 1. Determine the cluster info and user to use. At this point, we may or may not have a context. They are built based on the first hit in this chain. (run it twice, once for user, once for cluster) + 1. command line argument - `user` for user name and `cluster` for cluster name + 1. If context is present, then use the context's value + 1. Empty is allowed + 1. Determine the actual cluster info to use. At this point, we may or may not have a cluster info. Build each piece of the cluster info based on the chain (first hit wins): + 1. command line arguments - `server`, `api-version`, `certificate-authority`, and `insecure-skip-tls-verify` + 1. If cluster info is present and a value for the attribute is present, use it. + 1. If you don't have a server location, error. + 1. Determine the actual user info to use. User is built using the same rules as cluster info, EXCEPT that you can only have one authentication technique per user. + 1. Load precedence is 1) command line flag, 2) user fields from kubeconfig + 1. The command line flags are: `client-certificate`, `client-key`, `username`, `password`, and `token`. + 1. If there are two conflicting techniques, fail. + 1. For any information still missing, use default values and potentially prompt for authentication information + +## Manipulation of kubeconfig via `kubectl config ` + +In order to more easily manipulate kubeconfig files, there are a series of subcommands to `kubectl config` to help. +See [kubectl/kubectl_config.md](kubectl/kubectl_config) for help. + +### Example + +{% highlight console %} +$ kubectl config set-credentials myself --username=admin --password=secret +$ kubectl config set-cluster local-server --server=http://localhost:8080 +$ kubectl config set-context default-context --cluster=local-server --user=myself +$ kubectl config use-context default-context +$ kubectl config set contexts.default-context.namespace the-right-prefix +$ kubectl config view +{% endhighlight %} + +produces this output + +{% highlight yaml %} +apiVersion: v1 +clusters: +- cluster: + server: http://localhost:8080 + name: local-server +contexts: +- context: + cluster: local-server + namespace: the-right-prefix + user: myself + name: default-context +current-context: default-context +kind: Config +preferences: {} +users: +- name: myself + user: + password: secret + username: admin +{% endhighlight %} + +and a kubeconfig file that looks like this + +{% highlight yaml %} +apiVersion: v1 +clusters: +- cluster: + server: http://localhost:8080 + name: local-server +contexts: +- context: + cluster: local-server + namespace: the-right-prefix + user: myself + name: default-context +current-context: default-context +kind: Config +preferences: {} +users: +- name: myself + user: + password: secret + username: admin +{% endhighlight %} + +#### Commands for the example file + +{% highlight console %} +$ kubectl config set preferences.colors true +$ kubectl config set-cluster cow-cluster --server=http://cow.org:8080 --api-version=v1 +$ kubectl config set-cluster horse-cluster --server=https://horse.org:4443 --certificate-authority=path/to/my/cafile +$ kubectl config set-cluster pig-cluster --server=https://pig.org:443 --insecure-skip-tls-verify=true +$ kubectl config set-credentials blue-user --token=blue-token +$ kubectl config set-credentials green-user --client-certificate=path/to/my/client/cert --client-key=path/to/my/client/key +$ kubectl config set-context queen-anne-context --cluster=pig-cluster --user=black-user --namespace=saw-ns +$ kubectl config set-context federal-context --cluster=horse-cluster --user=green-user --namespace=chisel-ns +$ kubectl config use-context federal-context +{% endhighlight %} + +### Final notes for tying it all together + +So, tying this all together, a quick start to creating your own kubeconfig file: + +- Take a good look and understand how you're api-server is being launched: You need to know YOUR security requirements and policies before you can design a kubeconfig file for convenient authentication. + +- Replace the snippet above with information for your cluster's api-server endpoint. + +- Make sure your api-server is launched in such a way that at least one user (i.e. green-user) credentials are provided to it. You will of course have to look at api-server documentation in order to determine the current state-of-the-art in terms of providing authentication details. + + + + + diff --git a/v1.1/docs/user-guide/kubectl-overview.md b/v1.1/docs/user-guide/kubectl-overview.md index d70eaa7c99..c3d69055a0 100644 --- a/v1.1/docs/user-guide/kubectl-overview.md +++ b/v1.1/docs/user-guide/kubectl-overview.md @@ -1,32 +1,7 @@ --- title: "kubectl overview" --- - - -# kubectl overview - -Use this overview of the `kubectl` command line interface to help you start running commands against Kubernetes clusters. This overview quickly covers `kubectl` syntax, describes the command operations, and provides common examples. For details about each command, including all the supported flags and subcommands, see the [kubectl](kubectl/kubectl.html) reference documentation. - -**Table of contents:** - - -- [kubectl overview](#kubectl-overview) - - [Syntax](#syntax) - - [Operations](#operations) - - [Resource types](#resource-types) - - [Output options](#output-options) - - [Formatting output](#formatting-output) - - [Syntax](#syntax) - - [Example](#example) - - [Custom columns](#custom-columns) - - [Examples](#examples) - - [Sorting list objects](#sorting-list-objects) - - [Syntax](#syntax) - - [Example](#example) - - [Examples: Common operations](#examples-common-operations) - - [Next steps](#next-steps) - - +Use this overview of the `kubectl` command line interface to help you start running commands against Kubernetes clusters. This overview quickly covers `kubectl` syntax, describes the command operations, and provides common examples. For details about each command, including all the supported flags and subcommands, see the [kubectl](kubectl/kubectl) reference documentation. TODO: Auto-generate this file to ensure it's always in sync with any `kubectl` changes, see [#14177](http://pr.k8s.io/14177). @@ -35,9 +10,9 @@ TODO: Auto-generate this file to ensure it's always in sync with any `kubectl` c Use the following syntax to run `kubectl` commands from your terminal window: ``` -{% raw %} + kubectl [command] [TYPE] [NAME] [flags] -{% endraw %} + ``` where `command`, `TYPE`, `NAME`, and `flags` are: @@ -46,11 +21,11 @@ where `command`, `TYPE`, `NAME`, and `flags` are: * `TYPE`: Specifies the [resource type](#resource-types). Resource types are case-sensitive and you can specify the singular, plural, or abbreviated forms. For example, the following commands produce the same output: ``` -{% raw %} + $ kubectl get pod pod1 $ kubectl get pods pod1 $ kubectl get po pod1 -{% endraw %} + ``` * `NAME`: Specifies the name of the resource. Names are case-sensitive. If the name is omitted, details for all resources are displayed, for example `$ kubectl get pods`. @@ -62,7 +37,7 @@ where `command`, `TYPE`, `NAME`, and `flags` are: * To specify multiple resource types individually: `TYPE1/name1 TYPE1/name2 TYPE2/name3 TYPE<#>/name<#>`
    Example: `$ kubectl get pod/example-pod1 replicationcontroller/example-rc1` * To specify resources with one or more files: `-f file1 -f file2 -f file<#>` - [Use YAML rather than JSON](config-best-practices.html) since YAML tends to be more user-friendly, especially for configuration files.
    + [Use YAML rather than JSON](config-best-practices) since YAML tends to be more user-friendly, especially for configuration files.
    Example: `$ kubectl get pod -f ./pod.yaml` * `flags`: Specifies optional flags. For example, you can use the `-s` or `--server` flags to specify the address and port of the Kubernetes API server.
    **Important**: Flags that you specify from the command line override default values and any corresponding environment variables. @@ -101,7 +76,7 @@ Operation | Syntax | Description `stop` | `kubectl stop` | Deprecated: Instead, see `kubectl delete`. `version` | `kubectl version [--client] [flags]` | Display the Kubernetes version running on the client and server. -Remember: For more about command operations, see the [kubectl](kubectl/kubectl.html) reference documentation. +Remember: For more about command operations, see the [kubectl](kubectl/kubectl) reference documentation. ## Resource types @@ -128,7 +103,7 @@ Resource type | Abbreviated alias ## Output options -Use the following sections for information about how you can format or sort the output of certain commands. For details about which commands support the various output options, see the [kubectl](kubectl/kubectl.html) reference documentation. +Use the following sections for information about how you can format or sort the output of certain commands. For details about which commands support the various output options, see the [kubectl](kubectl/kubectl) reference documentation. ### Formatting output @@ -137,9 +112,9 @@ The default output format for all `kubectl` commands is the human readable plain #### Syntax ``` -{% raw %} + kubectl [command] [TYPE] [NAME] -o= -{% endraw %} + ``` Depending on the `kubectl` operation, the following output formats are supported: @@ -149,8 +124,8 @@ Output format | Description `-o=custom-columns=` | Print a table using a comma separated list of [custom columns](#custom-columns). `-o=custom-columns-file=` | Print a table using the [custom columns](#custom-columns) template in the `` file. `-o=json` | Output a JSON formatted API object. -`-o=jsonpath=