From a1b0c2e816714c51e81685a9b0d1b00be183511c Mon Sep 17 00:00:00 2001 From: Nick Veitch Date: Tue, 9 Apr 2019 03:31:03 +0100 Subject: [PATCH] Update Ubuntu docs (#13594) * update links to point to official docs * delete unmaintained content * Add page to retain url and direct readers to appropriate docs --- .../en/docs/getting-started-guides/ubuntu.md | 41 ++ .../getting-started-guides/ubuntu/_index.md | 75 ---- .../getting-started-guides/ubuntu/backups.md | 144 ------- .../ubuntu/decommissioning.md | 80 ---- .../getting-started-guides/ubuntu/glossary.md | 24 -- .../ubuntu/installation.md | 299 --------------- .../getting-started-guides/ubuntu/local.md | 81 ---- .../getting-started-guides/ubuntu/logging.md | 63 --- .../ubuntu/monitoring.md | 145 ------- .../ubuntu/networking.md | 63 --- .../ubuntu/operational-considerations.md | 165 -------- .../getting-started-guides/ubuntu/rancher.md | 360 ------------------ .../getting-started-guides/ubuntu/scaling.md | 87 ----- .../getting-started-guides/ubuntu/security.md | 36 -- .../getting-started-guides/ubuntu/storage.md | 95 ----- .../ubuntu/troubleshooting.md | 179 --------- .../getting-started-guides/ubuntu/upgrades.md | 164 -------- .../ubuntu/validation.md | 169 -------- content/en/docs/setup/pick-right-solution.md | 26 +- 19 files changed, 54 insertions(+), 2242 deletions(-) create mode 100644 content/en/docs/getting-started-guides/ubuntu.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/_index.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/backups.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/decommissioning.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/glossary.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/installation.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/local.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/logging.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/monitoring.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/networking.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/operational-considerations.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/rancher.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/scaling.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/security.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/storage.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/troubleshooting.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/upgrades.md delete mode 100644 content/en/docs/getting-started-guides/ubuntu/validation.md diff --git a/content/en/docs/getting-started-guides/ubuntu.md b/content/en/docs/getting-started-guides/ubuntu.md new file mode 100644 index 0000000000..ed77113f12 --- /dev/null +++ b/content/en/docs/getting-started-guides/ubuntu.md @@ -0,0 +1,41 @@ +--- +title: Kubernetes on Ubuntu +content_template: templates/concept +--- + +{{% capture overview %}} +There are multiple ways to run a Kubernetes cluster with Ubuntu on public and +private clouds, as well as bare metal. +{{% /capture %}} + +{{% capture body %}} +## The Charmed Distribution of Kubernetes(CDK) + +[CDK](https://www.ubuntu.com/cloud/kubernetes) is a distribution of Kubernetes +packaged as a bundle of *charms* for Juju, the open source application modeller. + +CDK is the latest version of Kubernetes with upstream binaries, packaged in a format +which makes it fast and easy to deploy. It supports various public +and private clouds including AWS, GCE, Azure, Joyent, OpenStack, VMware, Bare Metal +and localhost deployments. + +See the [Official documentation](https://www.ubuntu.com/cloud/kubernetes/docs) for +more information. + +## MicroK8s + +[MicroK8s](https://microk8s.io) is a minimal install of Kubernetes designed to run locally. +It can be installed on Ubuntu (or any snap enabled operating system) with the command: + +```shell +snap install microk8s --classic +``` + +Full documentation is available on the [MicroK8s website](https://microk8s.io/docs) + + + + + + +{{% /capture %}} diff --git a/content/en/docs/getting-started-guides/ubuntu/_index.md b/content/en/docs/getting-started-guides/ubuntu/_index.md deleted file mode 100644 index ed60790b67..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/_index.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -title: Kubernetes on Ubuntu -content_template: templates/concept ---- - -{{% capture overview %}} -There are multiple ways to run a Kubernetes cluster with Ubuntu. These pages explain how to deploy Kubernetes on Ubuntu on multiple public and private clouds, as well as bare metal. -{{% /capture %}} - -{{% capture body %}} -## Official Ubuntu Guides - -- [The Canonical Distribution of Kubernetes](https://www.ubuntu.com/cloud/kubernetes) - -The latest version of Kubernetes with upstream binaries. Supports AWS, GCE, Azure, Joyent, OpenStack, VMware, Bare Metal and localhost deployments. - -### Quick Start - -[conjure-up](http://conjure-up.io/) provides the quickest way to deploy Kubernetes on Ubuntu for multiple clouds and bare metal. It provides a user-friendly UI that prompts you for cloud credentials and configuration options - -Available for Ubuntu 16.04 and newer: - -``` -sudo snap install conjure-up --classic -# re-login may be required at that point if you just installed snap utility -conjure-up kubernetes -``` - -As well as Homebrew for macOS: - -``` -brew install conjure-up -conjure-up kubernetes -``` - -### Operational Guides - -These are more in-depth guides for users choosing to run Kubernetes in production: - - - [Installation](/docs/getting-started-guides/ubuntu/installation/) - - [Validation](/docs/getting-started-guides/ubuntu/validation/) - - [Backups](/docs/getting-started-guides/ubuntu/backups/) - - [Upgrades](/docs/getting-started-guides/ubuntu/upgrades/) - - [Scaling](/docs/getting-started-guides/ubuntu/scaling/) - - [Logging](/docs/getting-started-guides/ubuntu/logging/) - - [Monitoring](/docs/getting-started-guides/ubuntu/monitoring/) - - [Networking](/docs/getting-started-guides/ubuntu/networking/) - - [Security](/docs/getting-started-guides/ubuntu/security/) - - [Storage](/docs/getting-started-guides/ubuntu/storage/) - - [Troubleshooting](/docs/getting-started-guides/ubuntu/troubleshooting/) - - [Decommissioning](/docs/getting-started-guides/ubuntu/decommissioning/) - - [Operational Considerations](/docs/getting-started-guides/ubuntu/operational-considerations/) - - [Glossary](/docs/getting-started-guides/ubuntu/glossary/) - - [Authenticating with LDAP](https://www.ubuntu.com/kubernetes/docs/ldap) - -## Third-party Product Integrations - - - [Rancher](/docs/getting-started-guides/ubuntu/rancher/) - -## Developer Guides - - - [Localhost using LXD](/docs/getting-started-guides/ubuntu/local/) - -## Where to find us - -We're normally following the following Slack channels: - -- [kubernetes-users](https://kubernetes.slack.com/messages/kubernetes-users/) -- [kubernetes-novice](https://kubernetes.slack.com/messages/kubernetes-novice/) -- [sig-cluster-lifecycle](https://kubernetes.slack.com/messages/sig-cluster-lifecycle/) -- [sig-cluster-ops](https://kubernetes.slack.com/messages/sig-cluster-ops/) -- [sig-onprem](https://kubernetes.slack.com/messages/sig-onprem/) - -and we monitor the Kubernetes mailing lists. -{{% /capture %}} diff --git a/content/en/docs/getting-started-guides/ubuntu/backups.md b/content/en/docs/getting-started-guides/ubuntu/backups.md deleted file mode 100644 index 164949d24b..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/backups.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -title: Backups -content_template: templates/task ---- - -{{% capture overview %}} -The state of a Kubernetes cluster is kept in the etcd datastore. -This page shows how to backup and restore the etcd shipped with -the Canonical Distribution of Kubernetes. Backing up application specific data, -normally stored in a persistent volume, is outside the scope of this -document. -{{% /capture %}} - -{{% capture prerequisites %}} -This page assumes you have a working Juju deployed cluster. -{{% /capture %}} - -{{% capture steps %}} -## Snapshot etcd data - -The `snapshot` action of the etcd charm allows the operator to snapshot -a running cluster's data for use in cloning, -backing up, or migrating to a new cluster. - - juju run-action etcd/0 snapshot - -This will create a snapshot in `/home/ubuntu/etcd-snapshots` by default. - -## Restore etcd data - -The etcd charm is capable of restoring its data from a cluster-data snapshot -via the `restore` action. -This comes with caveats and a very specific path to restore a cluster: -The cluster must be in a state of only having a single member. So it's best to -deploy a new cluster using the etcd charm, without adding any additional units. - -``` -juju deploy etcd new-etcd -``` - -The above code snippet will deploy a single unit of etcd, as 'new-etcd' - -``` -juju run-action new-etcd/0 restore target=/mnt/etcd-backups -``` - -Once the restore action has completed, evaluate the cluster health. If the unit -is healthy, you may resume scaling the application to meet your needs. - -- **param** target: destination directory to save the existing data. - -- **param** skip-backup: Don't backup any existing data. - - -## Migrating an etcd cluster -Using the above snapshot and restore operations, migrating etcd is a fairly easy task. - -**Step 1:** Snapshot your existing cluster. This is encapsulated in the `snapshot` -action. - -``` -juju run-action etcd/0 snapshot -``` - -Results: - -``` -Action queued with id: b46d5d6f-5625-4320-8cda-b611c6ae580c -``` - -**Step 2:** Check the status of the action so you can grab the snapshot and verify -the sum. The `copy.cmd` result output is a copy/paste command for you to download -the exact snapshot that you just created. - -Download the snapshot archive from the unit that created the snapshot and verify -the sha256 sum - -``` -juju show-action-output b46d5d6f-5625-4320-8cda-b611c6ae580c -``` - -Results: - -``` -results: - copy: - cmd: juju scp etcd/0:/home/ubuntu/etcd-snapshots/etcd-snapshot-2016-11-09-02.41.47.tar.gz - . - snapshot: - path: /home/ubuntu/etcd-snapshots/etcd-snapshot-2016-11-09-02.41.47.tar.gz - sha256: 1dea04627812397c51ee87e313433f3102f617a9cab1d1b79698323f6459953d - size: 68K -status: completed -``` - -Copy the snapshot to the local disk and then check the sha256sum. - -``` -juju scp etcd/0:/home/ubuntu/etcd-snapshots/etcd-snapshot-2016-11-09-02.41.47.tar.gz . -sha256sum etcd-snapshot-2016-11-09-02.41.47.tar.gz -``` - -**Step 3:** Deploy the new cluster leader, and attach the snapshot: - -``` -juju deploy etcd new-etcd --resource snapshot=./etcd-snapshot-2016-11-09-02.41.47.tar.gz -``` - -**Step 4:** Reinitialize the master with the data from the resource we just attached -in step 3. - -``` -juju run-action new-etcd/0 restore -``` - - -{{% /capture %}} - -{{% capture discussion %}} - -## Known Limitations - -#### Loss of PKI warning - -If you destroy the leader - identified with the `*` text next to the unit number in status: -all TLS pki will be lost. No PKI migration occurs outside -of the units requesting and registering the certificates. - -{{< caution >}} -Mismanaging this configuration will result in locking yourself out of the cluster, -and can potentially break existing deployments in very strange ways relating to x509 -validation of certificates, which affects both servers and clients. -{{< /caution >}} - -#### Restoring from snapshot on a scaled cluster - -Restoring from a snapshot on a scaled cluster will result in a broken cluster. -Etcd performs clustering during unit turn-up, and state is stored in Etcd itself. -During the snapshot restore phase, a new cluster ID is initialized, and peers -are dropped from the snapshot state to enable snapshot restoration. Please -follow the migration instructions above in the restore action description. -{{% /capture %}} - - diff --git a/content/en/docs/getting-started-guides/ubuntu/decommissioning.md b/content/en/docs/getting-started-guides/ubuntu/decommissioning.md deleted file mode 100644 index 04f5210251..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/decommissioning.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: Decommissioning -content_template: templates/task ---- - -{{% capture overview %}} -This page shows you how to properly decommission a cluster. -{{% /capture %}} - - -{{% capture prerequisites %}} -This page assumes you have a working Juju deployed cluster. - -{{< warning >}} -By the time you've reached this step you should have backed up your workloads and pertinent data; this section is for the complete destruction of a cluster. -{{< /warning >}} - -{{% /capture %}} - -{{% capture steps %}} -## Destroy the Juju model -It is recommended to deploy individual Kubernetes clusters in their own models, so that there is a clean separation between environments. To remove a cluster first find out which model it's in with `juju list-models`. The controller reserves an `admin` model for itself. If you have chosen to not name your model it might show up as `default`. - -``` -$ juju list-models -Controller: aws-us-east-2 - -Model Cloud/Region Status Machines Cores Access Last connection -controller aws/us-east-2 available 1 2 admin just now -my-kubernetes-cluster* aws/us-east-2 available 12 22 admin 2 minutes ago -``` - -You can then destroy the model, which will in turn destroy the cluster inside of it: - - juju destroy-model my-kubernetes-cluster - -``` -$ juju destroy-model my-kubernetes-cluster -WARNING! This command will destroy the "my-kubernetes-cluster" model. -This includes all machines, applications, data and other resources. - -Continue [y/N]? y -Destroying model -Waiting on model to be removed, 12 machine(s), 10 application(s)... -Waiting on model to be removed, 12 machine(s), 9 application(s)... -Waiting on model to be removed, 12 machine(s), 8 application(s)... -Waiting on model to be removed, 12 machine(s), 7 application(s)... -Waiting on model to be removed, 12 machine(s)... -Waiting on model to be removed... -$ -``` - -This will destroy and decommission all nodes. You can confirm all nodes are destroyed by running `juju status`. - -If you're using a public cloud this will terminate the instances. If you're on bare metal using MAAS this will release the nodes, optionally wipe the disk, power off the machines, and return them to available pool of machines to deploy from. - -## Cleaning up the Controller - -If you're not using the controller for anything else, you will also need to remove the controller instance: - -``` -$ juju list-controllers -Use --refresh flag with this command to see the latest information. - -Controller Model User Access Cloud/Region Models Machines HA Version -aws-us-east-2* - admin superuser aws/us-east-2 2 1 none 2.0.1 - -$ juju destroy-controller aws-us-east-2 -WARNING! This command will destroy the "aws-us-east-2" controller. -This includes all machines, applications, data and other resources. - -Continue? (y/N):y -Destroying controller -Waiting for hosted model resources to be reclaimed -All hosted models reclaimed, cleaning up controller machines -$ -``` -{{% /capture %}} - - diff --git a/content/en/docs/getting-started-guides/ubuntu/glossary.md b/content/en/docs/getting-started-guides/ubuntu/glossary.md deleted file mode 100644 index 391190eebf..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/glossary.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: Glossary and Terminology -content_template: templates/concept ---- - -{{% capture overview %}} -This page explains some of the terminology used in deploying Kubernetes with Juju. -{{% /capture %}} - -{{% capture body %}} - - -**controller** - The management node of a cloud environment. Typically you have one controller per cloud region, or more in HA environments. The controller is responsible for managing all subsequent models in a given environment. It contains the Juju API server and its underlying database. - -**model** - A collection of charms and their relationships that define a deployment. This includes machines and units. A controller can host multiple models. It is recommended to separate Kubernetes clusters into individual models for management and isolation reasons. - -**charm** - The definition of a service, including its metadata, dependencies with other services, required packages, and application management logic. It contains all the operational knowledge of deploying a Kubernetes cluster. Included charm examples are `kubernetes-core`, `easyrsa`, `flannel`, and `etcd`. - -**unit** - A given instance of a service. These may or may not use up a whole machine, and may be colocated on the same machine. So for example you might have a `kubernetes-worker`, and `etcd`, and `easyrsa` units running on a single machine, but they are three distinct units of different services. - -**machine** - A physical node, these can either be bare metal nodes, or virtual machines provided by a cloud. -{{% /capture %}} - - diff --git a/content/en/docs/getting-started-guides/ubuntu/installation.md b/content/en/docs/getting-started-guides/ubuntu/installation.md deleted file mode 100644 index eb01d37fbc..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/installation.md +++ /dev/null @@ -1,299 +0,0 @@ ---- -reviewers: -- caesarxuchao -- erictune -title: Setting up Kubernetes with Juju -content_template: templates/task ---- - -{{% capture overview %}} -Ubuntu 16.04 introduced the [Canonical Distribution of Kubernetes](https://www.ubuntu.com/cloud/kubernetes), a pure upstream distribution of Kubernetes designed for production usage. This page shows you how to deploy a cluster. -{{% /capture %}} - -{{% capture prerequisites %}} -- A working [Juju client](https://jujucharms.com/docs/2.3/reference-install); this does not have to be a Linux machine, it can also be Windows or macOS. -- A [supported cloud](#cloud-compatibility). - - Bare Metal deployments are supported via [MAAS](http://maas.io). Refer to the [MAAS documentation](http://maas.io/docs/) for configuration instructions. - - OpenStack deployments are currently only tested on Icehouse and newer. -- One of the following: - - Network access to the following domains - - *.jujucharms.com - - gcr.io - - github.com - - Access to an Ubuntu mirror (public or private) - - Offline deployment prepared with [these](https://github.com/juju-solutions/bundle-canonical-kubernetes/wiki/Running-CDK-in-a-restricted-environment) instructions. -{{% /capture %}} - - -{{% capture steps %}} -## Deployment overview -Out of the box the deployment comes with the following components on 9 machines: - -- Kubernetes (automated deployment, operations, and scaling) - - Four node Kubernetes cluster with one master and three worker nodes. - - TLS used for communication between units for security. - - Flannel Software Defined Network (SDN) plugin - - A load balancer for HA kubernetes-master (Experimental) - - Optional Ingress Controller (on worker) - - Optional Dashboard addon (on master) including Heapster for cluster monitoring -- EasyRSA - - Performs the role of a certificate authority serving self signed certificates - to the requesting units of the cluster. -- ETCD (distributed key value store) - - Three unit cluster for reliability. - -The Juju Kubernetes work is curated by the Big Software team at [Canonical Ltd](https://www.canonical.com/), -let us know how we are doing. If you find any problems please open an -[issue on our tracker](https://github.com/juju-solutions/bundle-canonical-kubernetes) -so we can find them. - -## Support Level - -IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level --------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -Amazon Web Services (AWS) | Juju | Ubuntu | flannel, calico* | [docs](/docs/getting-started-guides/ubuntu) | | [Commercial](https://ubuntu.com/cloud/kubernetes), [Community](https://github.com/juju-solutions/bundle-kubernetes-core) -OpenStack | Juju | Ubuntu | flannel, calico | [docs](/docs/getting-started-guides/ubuntu) | | [Commercial](https://ubuntu.com/cloud/kubernetes), [Community](https://github.com/juju-solutions/bundle-kubernetes-core) -Microsoft Azure | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu) | | [Commercial](https://ubuntu.com/cloud/kubernetes), [Community](https://github.com/juju-solutions/bundle-kubernetes-core) -Google Compute Engine (GCE) | Juju | Ubuntu | flannel, calico | [docs](/docs/getting-started-guides/ubuntu) | | [Commercial](https://ubuntu.com/cloud/kubernetes), [Community](https://github.com/juju-solutions/bundle-kubernetes-core) -Joyent | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu) | | [Commercial](https://ubuntu.com/cloud/kubernetes), [Community](https://github.com/juju-solutions/bundle-kubernetes-core) -Rackspace | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu) | | [Commercial](https://ubuntu.com/cloud/kubernetes), [Community](https://github.com/juju-solutions/bundle-kubernetes-core) -VMware vSphere | Juju | Ubuntu | flannel, calico | [docs](/docs/getting-started-guides/ubuntu) | | [Commercial](https://ubuntu.com/cloud/kubernetes), [Community](https://github.com/juju-solutions/bundle-kubernetes-core) -Bare Metal (MAAS) | Juju | Ubuntu | flannel, calico | [docs](/docs/getting-started-guides/ubuntu) | | [Commercial](https://ubuntu.com/cloud/kubernetes), [Community](https://github.com/juju-solutions/bundle-kubernetes-core) - - -For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. - -## Installation options - -You can launch a cluster in one of two ways: [conjure-up](#conjure-up) or [juju deploy](#juju-deploy). Conjure-up is just a convenience wrapper over juju and simplifies the installation. As such, it is the preferred method of install. - -Deployment of the cluster is [supported on a wide variety of public clouds](#cloud-compatibility), private OpenStack clouds, or raw bare metal clusters. Bare metal deployments are supported via [MAAS](http://maas.io/). - -## Conjure-up -To install Kubernetes with conjure-up, you need only to run the following commands and then follow the prompts: - -``` -sudo snap install conjure-up --classic -conjure-up kubernetes -``` -## Juju deploy - -### Configure Juju to use your cloud provider - -After deciding which cloud to deploy to, follow the [cloud setup page](https://jujucharms.com/docs/devel/getting-started) to configure deploying to that cloud. - -Load your [cloud credentials](https://jujucharms.com/docs/2.3/credentials) for each -cloud provider you would like to use. - -In this example - -``` -juju add-credential aws -credential name: my_credentials -select auth-type [userpass, oauth, etc]: userpass -enter username: jorge -enter password: ******* -``` - -You can also just auto load credentials for popular clouds with the `juju autoload-credentials` command, which will auto import your credentials from the default files and environment variables for each cloud. - -Next we need to bootstrap a controller to manage the cluster. You need to define the cloud you want to bootstrap on, the region, and then any name for your controller node: - -``` -juju update-clouds # This command ensures all the latest regions are up to date on your client -juju bootstrap aws/us-east-2 -``` -or, another example, this time on Azure: - -``` -juju bootstrap azure/westus2 -``` - -If you receive this error, it is likely that the default Azure VM size (Standard D1 v2 [1 vcpu, 3.5 GB memory]) is not available in the Azure location: -``` -ERROR failed to bootstrap model: instance provisioning failed (Failed) -``` - - -You will need a controller node for each cloud or region you are deploying to. See the [controller documentation](https://jujucharms.com/docs/2.3/controllers) for more information. - -Note that each controller can host multiple Kubernetes clusters in a given cloud or region. - -### Launch a Kubernetes cluster - -The following command will deploy the initial 9-node starter cluster. The speed of execution is very dependent of the performance of the cloud you're deploying to: - -``` -juju deploy canonical-kubernetes -``` - -After this command executes the cloud will then launch instances and begin the deployment process. - -## Monitor deployment - -The `juju status` command provides information about each unit in the cluster. Use the `watch -c juju status --color` command to get a real-time view of the cluster as it deploys. When all the states are green and "Idle", the cluster is ready to be used: - - juju status - -Output: - -``` -Model Controller Cloud/Region Version SLA -conjure-canonical-kubern-f48 conjure-up-aws-650 aws/us-east-2 2.3.2 unsupported - -App Version Status Scale Charm Store Rev OS Notes -easyrsa 3.0.1 active 1 easyrsa jujucharms 27 ubuntu -etcd 2.3.8 active 3 etcd jujucharms 63 ubuntu -flannel 0.9.1 active 4 flannel jujucharms 40 ubuntu -kubeapi-load-balancer 1.10.3 active 1 kubeapi-load-balancer jujucharms 43 ubuntu exposed -kubernetes-master 1.9.3 active 1 kubernetes-master jujucharms 13 ubuntu -kubernetes-worker 1.9.3 active 3 kubernetes-worker jujucharms 81 ubuntu exposed - -Unit Workload Agent Machine Public address Ports Message -easyrsa/0* active idle 3 18.219.190.99 Certificate Authority connected. -etcd/0 active idle 5 18.219.56.23 2379/tcp Healthy with 3 known peers -etcd/1* active idle 0 18.219.212.151 2379/tcp Healthy with 3 known peers -etcd/2 active idle 6 13.59.240.210 2379/tcp Healthy with 3 known peers -kubeapi-load-balancer/0* active idle 1 18.222.61.65 443/tcp Loadbalancer ready. -kubernetes-master/0* active idle 4 18.219.105.220 6443/tcp Kubernetes master running. - flannel/3 active idle 18.219.105.220 Flannel subnet 10.1.78.1/24 -kubernetes-worker/0 active idle 2 18.219.221.98 80/tcp,443/tcp Kubernetes worker running. - flannel/1 active idle 18.219.221.98 Flannel subnet 10.1.38.1/24 -kubernetes-worker/1* active idle 7 18.219.249.103 80/tcp,443/tcp Kubernetes worker running. - flannel/2 active idle 18.219.249.103 Flannel subnet 10.1.68.1/24 -kubernetes-worker/2 active idle 8 52.15.89.16 80/tcp,443/tcp Kubernetes worker running. - flannel/0* active idle 52.15.89.16 Flannel subnet 10.1.73.1/24 - -Machine State DNS Inst id Series AZ Message -0 started 18.219.212.151 i-065eab4eabc691b25 xenial us-east-2a running -1 started 18.222.61.65 i-0b332955f028d6281 xenial us-east-2b running -2 started 18.219.221.98 i-0879ef1ed95b569bc xenial us-east-2a running -3 started 18.219.190.99 i-08a7b364fc008fc85 xenial us-east-2c running -4 started 18.219.105.220 i-0f92d3420b01085af xenial us-east-2a running -5 started 18.219.56.23 i-0271f6448cebae352 xenial us-east-2c running -6 started 13.59.240.210 i-0789ef5837e0669b3 xenial us-east-2b running -7 started 18.219.249.103 i-02f110b0ab042f7ac xenial us-east-2b running -8 started 52.15.89.16 i-086852bf1bee63d4e xenial us-east-2c running - -Relation provider Requirer Interface Type Message -easyrsa:client etcd:certificates tls-certificates regular -easyrsa:client kubeapi-load-balancer:certificates tls-certificates regular -easyrsa:client kubernetes-master:certificates tls-certificates regular -easyrsa:client kubernetes-worker:certificates tls-certificates regular -etcd:cluster etcd:cluster etcd peer -etcd:db flannel:etcd etcd regular -etcd:db kubernetes-master:etcd etcd regular -kubeapi-load-balancer:loadbalancer kubernetes-master:loadbalancer public-address regular -kubeapi-load-balancer:website kubernetes-worker:kube-api-endpoint http regular -kubernetes-master:cni flannel:cni kubernetes-cni subordinate -kubernetes-master:kube-api-endpoint kubeapi-load-balancer:apiserver http regular -kubernetes-master:kube-control kubernetes-worker:kube-control kube-control regular -kubernetes-worker:cni flannel:cni kubernetes-cni subordinate -``` - -## Interacting with the cluster - -After the cluster is deployed you may assume control over the cluster from any kubernetes-master, or kubernetes-worker node. - -If you didn't use conjure-up, you will first need to download the credentials and client application to your local workstation: - -Create the kubectl config directory. - -``` -mkdir -p ~/.kube -``` -Copy the kubeconfig file to the default location. - -``` -juju scp kubernetes-master/0:/home/ubuntu/config ~/.kube/config -``` - -The next step is to install the kubectl client on your local machine. The recommended way to do this on Ubuntu is using the kubectl snap ([/docs/tasks/tools/install-kubectl/#install-with-snap-on-ubuntu](/docs/tasks/tools/install-kubectl/#install-with-snap-on-ubuntu)). - -The following command should be run on the machine you wish to use to control the kubernetes cluster: - -``` -sudo snap install kubectl --classic -``` - -This will install and deploy the kubectl binary. You may need to restart your terminal as your $PATH may have been updated. - -Query the cluster: - - kubectl cluster-info - -Output: - -``` -Kubernetes master is running at https://52.15.104.227:443 -Heapster is running at https://52.15.104.227:443/api/v1/namespaces/kube-system/services/heapster/proxy -KubeDNS is running at https://52.15.104.227:443/api/v1/namespaces/kube-system/services/kube-dns/proxy -Grafana is running at https://52.15.104.227:443/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy -InfluxDB is running at https://52.15.104.227:443/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy -``` - -Congratulations, you've now set up a Kubernetes cluster! - -## Scale up cluster - -Want larger Kubernetes nodes? It is easy to request different sizes of cloud -resources from Juju by using **constraints**. You can increase the amount of -CPU or memory (RAM) in any of the systems requested by Juju. This allows you -to fine tune the Kubernetes cluster to fit your workload. Use flags on the -bootstrap command or as a separate `juju constraints` command. Look to the -[Juju documentation for machine](https://jujucharms.com/docs/2.3/charms-constraints) -details. - -## Scale out cluster - -Need more workers? We just add more units: - -```shell -juju add-unit kubernetes-worker -``` - -Or multiple units at one time: - -```shell -juju add-unit -n3 kubernetes-worker -``` -You can also ask for specific instance types or other machine-specific constraints. See the [constraints documentation](https://jujucharms.com/docs/stable/reference-constraints) for more information. Here are some examples, note that generic constraints such as `cores` and `mem` are more portable between clouds. In this case we'll ask for a specific instance type from AWS: - -```shell -juju set-constraints kubernetes-worker instance-type=c4.large -juju add-unit kubernetes-worker -``` - -You can also scale the etcd charm for more fault tolerant key/value storage: - -```shell -juju add-unit -n3 etcd -``` -It is strongly recommended to run an odd number of units for quorum. - -## Tear down cluster - -If you used conjure-up to create your cluster, you can tear it down with `conjure-down`. If you used juju directly, you can tear it down by destroying the Juju model or the controller. Use the `juju switch` command to get the current controller name: - -```shell -juju switch -juju destroy-controller $controllername --destroy-all-models -``` -This will shutdown and terminate all running instances on that cloud. -{{% /capture %}} - -{{% capture discussion %}} -## More Info - -The Ubuntu Kubernetes deployment uses open-source operations, or operations as code, known as charms. These charms are assembled from layers which keeps the code smaller and more focused on the operations of just Kubernetes and its components. - -The Kubernetes layer and bundles can be found in the `kubernetes` -project on github.com: - - - [Bundle location](https://git.k8s.io/kubernetes/cluster/juju/bundles) - - [Kubernetes charm layer location](https://git.k8s.io/kubernetes/cluster/juju/layers) - - [Canonical Kubernetes home](https://jujucharms.com/kubernetes) - - [Main issue tracker](https://github.com/juju-solutions/bundle-canonical-kubernetes) - -Feature requests, bug reports, pull requests and feedback are appreciated. -{{% /capture %}} diff --git a/content/en/docs/getting-started-guides/ubuntu/local.md b/content/en/docs/getting-started-guides/ubuntu/local.md deleted file mode 100644 index 49e5e64f74..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/local.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -title: Local Kubernetes development with LXD -content_template: templates/task ---- - -{{% capture overview %}} -Running Kubernetes locally has obvious development advantages, such as lower cost and faster iteration than constantly deploying and tearing down clusters on a public cloud. Ideally, a Kubernetes developer can spawn all necessary nodes inside local containers and test new configurations as they are committed. This page will show you how to deploy a cluster to LXD containers on a local machine. -{{% /capture %}} - -The purpose of using [LXD](https://linuxcontainers.org/lxd/) on a local machine is to emulate the same deployment that a user would use in a cloud or bare metal. Each node is treated as a machine, with the same characteristics as production. Each node is a separate container, which runs Docker containers and `kubectl` inside (see [Cluster Intro](/docs/tutorials/kubernetes-basics/cluster-intro/) for more info). - -{{% capture prerequisites %}} -Install [conjure-up](http://conjure-up.io/), a tool for deploying big software. -Add the current user to the `lxd` user group. - -``` -sudo snap install conjure-up --classic -sudo usermod -a -G lxd $(whoami) -``` - -If you have never run or configured lxd before, you will need to run the following -command to set up the default storage pool and the network bridge required: - -``` -sudo lxd init -``` - -If a bridge named `lxdbr0` already exists, you can tell the init config tool to use it. - -Note: If conjure-up asks you to "Setup an IPv6 subnet" with LXD, answer NO. IPv6 with Juju/LXD is currently unsupported. -If you already have a bridge configured, e.g. `lxdbr0`, [disable IPv6 on the bridge](https://docs.conjure-up.io/stable/en/troubleshoot#common-problems), otherwise you won't be able to choose it. -{{% /capture %}} - -{{% capture steps %}} -## Deploying Kubernetes - -Start the deployment with: - - conjure-up kubernetes - -For this walkthrough we are going to create a new controller - select the `localhost` Cloud type: - -![Select Cloud](/images/docs/ubuntu/00-select-cloud.png) - -Deploy the applications: - -![Deploy Applications](/images/docs/ubuntu/01-deploy.png) - -Wait for Juju bootstrap to finish: - -![Bootstrap](/images/docs/ubuntu/02-bootstrap.png) - -Wait for our Applications to be fully deployed: - -![Waiting](/images/docs/ubuntu/03-waiting.png) - -Run the final post-processing steps to automatically configure your Kubernetes environment: - -![Postprocessing](/images/docs/ubuntu/04-postprocessing.png) - -Review the final summary screen: - -![Final Summary](/images/docs/ubuntu/05-final-summary.png) - -## Accessing the Cluster - -You can access your Kubernetes cluster by running the following: - - - kubectl --kubeconfig=~/.kube/config - - -Or if you've already run this once it'll create a new config file as shown in the summary screen. - - - kubectl --kubeconfig=~/.kube/config.conjure-up - -{{% /capture %}} - - - diff --git a/content/en/docs/getting-started-guides/ubuntu/logging.md b/content/en/docs/getting-started-guides/ubuntu/logging.md deleted file mode 100644 index e691731d29..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/logging.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -title: Logging -content_template: templates/task ---- - -{{% capture overview %}} -This page will explain how logging works within a Juju deployed cluster. -{{% /capture %}} - -{{% capture prerequisites %}} -This page assumes you have a working Juju deployed cluster. -{{% /capture %}} - -{{% capture steps %}} -## Agent Logging - -The `juju debug-log` will show all of the consolidated logs of all the Juju agents running on each node of the cluster. This can be useful for finding out why a specific node hasn't deployed or is in an error state. These agent logs are located in `/var/lib/juju/agents` on every node. - -See the [Juju documentation](https://jujucharms.com/docs/stable/troubleshooting-logs) for more information. - - -## Managing log verbosity - -Log verbosity in Juju is set at the model level. You can adjust it at any time: - -``` -juju add-model k8s-development --config logging-config='=DEBUG;unit=DEBUG' -``` - -and later on your k8s-production model - -``` -juju model-config -m k8s-production logging-config='=ERROR;unit=ERROR' -``` - -In addition, the jujud daemon is started in debug mode by default on all controllers. To remove that behavior edit ```/var/lib/juju/init/jujud-machine-0/exec-start.sh``` on the controller node and comment the ```--debug``` section. - -It then contains: - -``` -#!/usr/bin/env bash - -# Set up logging. -touch '/var/log/juju/machine-0.log' -chown syslog:syslog '/var/log/juju/machine-0.log' -chmod 0600 '/var/log/juju/machine-0.log' -exec >> '/var/log/juju/machine-0.log' -exec 2>&1 - -# Run the script. -'/var/lib/juju/tools/machine-0/jujud' machine --data-dir '/var/lib/juju' --machine-id 0 # --debug -``` - -Then restart the service with: - -``` -sudo systemctl restart jujud-machine-0.service -``` - -See the [official documentation](https://jujucharms.com/docs/stable/models-config) for more information about logging and other model settings in Juju. -{{% /capture %}} - - diff --git a/content/en/docs/getting-started-guides/ubuntu/monitoring.md b/content/en/docs/getting-started-guides/ubuntu/monitoring.md deleted file mode 100644 index 110de5cb13..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/monitoring.md +++ /dev/null @@ -1,145 +0,0 @@ ---- -title: Monitoring -content_template: templates/task ---- - -{{% capture overview %}} -This page shows how to connect various logging solutions to a Juju deployed cluster. -{{% /capture %}} - -{{% capture prerequisites %}} -This page assumes you have a working Juju deployed cluster. -{{% /capture %}} - -{{% capture steps %}} -## Connecting Datadog - -Datadog is a SaaS offering which includes support for a range of integrations, including Kubernetes and ETCD. While the solution is SAAS/Commercial, they include a Free tier which is supported with the following method. To deploy a full Kubernetes stack with Datadog out of the box, do: - -``` -juju deploy canonical-kubernetes-datadog -``` - -### Installation of Datadog - -To start, deploy the latest version Datadog from the Charm Store: - -``` -juju deploy datadog -``` - -Configure Datadog with your api-key, found in the [Datadog dashboard](). Replace `XXXX` with your API key. - -``` -juju configure datadog api-key=XXXX -``` - -Finally, attach `datadog` to all applications you wish to monitor. For example, kubernetes-master, kubernetes-worker, and etcd: - -``` -juju add-relation datadog kubernetes-worker -juju add-relation datadog kubernetes-master -juju add-relation datadog etcd -``` - -## Connecting Elastic stack - -The Elastic stack, formally "ELK" stack, refers to Elastic Search and the suite of tools to facilitate log aggregation, monitoring, and dashboarding. To deploy a full Kubernetes stack with elastic out of the box, do: - -``` -juju deploy canonical-kubernetes-elastic -``` - -### New install of ElasticSearch - -To start, deploy the latest version of ElasticSearch, Kibana, Filebeat, and Topbeat from the Charm Store: - -This can be done in one command as: - -``` -juju deploy beats-core -``` - -However, if you wish to customize the deployment, or proceed manually, the following commands can be issued: - -``` -juju deploy elasticsearch -juju deploy kibana -juju deploy filebeat -juju deploy topbeat - -juju add-relation elasticsearch kibana -juju add-relation elasticsearch topbeat -juju add-relation elasticsearch filebeat -``` - -Finally, connect filebeat and topbeat to all applications you wish to monitor. For example, kubernetes-master and kubernetes-worker: - -``` -juju add-relation kubernetes-master topbeat -juju add-relation kubernetes-master filebeat -juju add-relation kubernetes-worker topbeat -juju add-relation kubernetes-worker filebeat -``` - -### Existing ElasticSearch cluster - -In the event an ElasticSearch cluster already exists, the following can be used to connect and leverage it instead of creating a new, separate, cluster. First deploy the two beats, filebeat and topbeat - -``` -juju deploy filebeat -juju deploy topbeat -``` - -Configure both filebeat and topbeat to connect to your ElasticSearch cluster, replacing `255.255.255.255` with the IP address in your setup. - -``` -juju configure filebeat elasticsearch=255.255.255.255 -juju configure topbeat elasticsearch=255.255.255.255 -``` - -Follow the above instructions on connect topbeat and filebeat to the applications you wish to monitor. - - -## Connecting Nagios - -Nagios utilizes the Nagios Remote Plugin Executor protocol (NRPE protocol) as an agent on each node to derive machine level details of the health and applications. - -### New install of Nagios - -To start, deploy the latest version of the Nagios and NRPE charms from the store: - -``` -juju deploy nagios -juju deploy nrpe -``` - -Connect Nagios to NRPE - -``` -juju add-relation nagios nrpe -``` - -Finally, add NRPE to all applications deployed that you wish to monitor, for example `kubernetes-master`, `kubernetes-worker`, `etcd`, `easyrsa`, and `kubeapi-load-balancer`. - -``` -juju add-relation nrpe kubernetes-master -juju add-relation nrpe kubernetes-worker -juju add-relation nrpe etcd -juju add-relation nrpe easyrsa -juju add-relation nrpe kubeapi-load-balancer -``` - -### Existing install of Nagios - -If you already have an existing Nagios installation, the `nrpe-external-master` charm can be used instead. This will allow you to supply configuration options that map your existing external Nagios installation to NRPE. Replace `255.255.255.255` with the IP address of the nagios instance. - -``` -juju deploy nrpe-external-master -juju configure nrpe-external-master nagios_master=255.255.255.255 -``` - -Once configured, connect nrpe-external-master as outlined above. -{{% /capture %}} - - diff --git a/content/en/docs/getting-started-guides/ubuntu/networking.md b/content/en/docs/getting-started-guides/ubuntu/networking.md deleted file mode 100644 index 41c43cc7b1..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/networking.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -title: Networking -content_template: templates/task ---- - -{{% capture overview %}} -Kubernetes supports the [Container Network Interface (CNI)](https://github.com/containernetworking/cni). -This is a network plugin architecture that allows you to use whatever -Kubernetes-friendly SDN you want. Currently this means support for Flannel and Canal. - -This page shows how the various network portions of a cluster work and how to configure them. -{{% /capture %}} -{{% capture prerequisites %}} -This page assumes you have a working Juju deployed cluster. - -{{< note >}} -Note that if you deploy a cluster via conjure-up or the CDK bundles, manually deploying CNI plugins is unnecessary. -{{< /note >}} -{{% /capture %}} - - -{{% capture steps %}} -The CNI charms are [subordinates](https://jujucharms.com/docs/stable/authors-subordinate-applications). -These charms will require a principal charm that implements the `kubernetes-cni` interface in order to properly deploy. - -## Flannel - -``` -juju deploy flannel -juju add-relation flannel kubernetes-master -juju add-relation flannel kubernetes-worker -juju add-relation flannel etcd -``` - -## Canal - -``` -juju deploy canal -juju add-relation canal kubernetes-master -juju add-relation canal kubernetes-worker -juju add-relation canal etcd -``` - -### Configuration - -**iface** The interface to configure the flannel or canal SDN binding. If this value is -empty string or undefined the code will attempt to find the default network -adapter similar to the following command: - -```bash -route | grep default | head -n 1 | awk {'print $8'} -``` - -**cidr** The network range to configure the flannel or canal SDN to declare when -establishing networking setup with etcd. Ensure this network range is not active -on layers 2/3 you're deploying to, as it will cause collisions and odd behavior -if care is not taken when selecting a good CIDR range to assign to flannel. It's -also good practice to ensure you allot yourself a large enough IP range to support -how large your cluster will potentially scale. Class A IP ranges with /24 are -a good option. -{{% /capture %}} - - diff --git a/content/en/docs/getting-started-guides/ubuntu/operational-considerations.md b/content/en/docs/getting-started-guides/ubuntu/operational-considerations.md deleted file mode 100644 index 18c0f12264..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/operational-considerations.md +++ /dev/null @@ -1,165 +0,0 @@ ---- -title: Operational Considerations -content_template: templates/task ---- - -{{% capture overview %}} -This page gives recommendations and hints for people managing long lived clusters -{{% /capture %}} -{{% capture prerequisites %}} -This page assumes you understand the basics of Juju and Kubernetes. -{{% /capture %}} - -{{% capture steps %}} - -## Managing Juju - -### Sizing your controller node - -The Juju Controller: - -* requires about 2 to 2.5GB RAM to operate. -* uses a MongoDB database as a storage backend for the configuration and state of the cluster. This database can grow significantly, and can also be the biggest consumer of CPU cycles on the instance -* aggregates and stores the log data of all services and units. Therefore, significant storage is needed for long lived models. If your intention is to keep the cluster running, make sure to provision at least 64GB for the logs. - -To bootstrap a controller with constraints run the following command: - -``` -juju bootstrap --constraints "mem=8GB cpu-cores=4 root-disk=128G" -``` - -Juju will select the cheapest instance type matching your constraints on your target cloud. You can also use the ```instance-type``` constraint in conjunction with ```root-disk``` for strict control. For more information about the constraints available, refer to the [official documentation](https://jujucharms.com/docs/stable/reference-constraints) - -Additional information about logging can be found in the [logging section](/docs/getting-started-guides/ubuntu/logging) - -### SSHing into the Controller Node - -By default, Juju will create a pair of SSH keys that it will use to automate the connection to units. They are stored on the client node in ```~/.local/share/juju/ssh/``` - -After deployment, Juju Controller is a "silent unit" that acts as a proxy between the client and the deployed applications. Nevertheless it can be useful to SSH into it. - -First you need to understand your environment, especially if you run several Juju models and controllers. Run - -``` -juju list-models --all -$ juju models --all -Controller: k8s - -Model Cloud/Region Status Machines Cores Access Last connection -admin/controller lxd/localhost available 1 - admin just now -admin/default lxd/localhost available 0 - admin 2017-01-23 -admin/whale* lxd/localhost available 6 - admin 3 minutes ago -``` - -The first line ```Controller: k8s``` refers to how you bootstrapped. - -Then you will see 2, 3 or more models listed below. - -* admin/controller is the default model that hosts all controller units of juju -* admin/default is created by default as the primary model to host the user application, such as the Kubernetes cluster -* admin/whale is an additional model created if you use conjure-up as an overlay on top of Juju. - -Now to ssh into a controller node, you first ask Juju to switch context, then ssh as you would with a normal unit: - -``` -juju switch controller -``` - -At this stage, you can query the controller model as well: - -``` -juju status -Model Controller Cloud/Region Version -controller k8s lxd/localhost 2.0.2 - -App Version Status Scale Charm Store Rev OS Notes - -Unit Workload Agent Machine Public address Ports Message - -Machine State DNS Inst id Series AZ -0 started 10.191.22.15 juju-2a5ed8-0 xenial -``` - -Note that if you had bootstrapped in HA mode, you would see several machines listed. - -Now ssh-ing into the controller follows the same semantic as classic Juju commands: - -``` -$ juju ssh 0 -Welcome to Ubuntu 16.04.1 LTS (GNU/Linux 4.8.0-34-generic x86_64) - - * Documentation: https://help.ubuntu.com - * Management: https://landscape.canonical.com - * Support: https://ubuntu.com/advantage - - Get cloud support with Ubuntu Advantage Cloud Guest: - http://www.ubuntu.com/business/services/cloud - -0 packages can be updated. -0 updates are security updates. - - -Last login: Tue Jan 24 16:38:13 2017 from 10.191.22.1 -ubuntu@juju-2a5ed8-0:~$ -``` - -When you are done and want to come back to your initial model, exit the controller and - - -Then if you need to switch back to your cluster and ssh into the units, run - -``` -juju switch default -``` - -## Managing your Kubernetes cluster - -### Running privileged containers - -By default, juju-deployed clusters only allow running privileged containers on nodes with GPUs. -If you need privileged containers on other nodes, you have to enable the ```allow-privileged``` config on both -kubernetes-master and kubernetes-worker: - -``` -juju config kubernetes-master allow-privileged=true -juju config kubernetes-worker allow-privileged=true -``` - -### Private registry - -With the registry action, you can easily create a private Docker registry that -uses TLS authentication. However, note that a registry deployed with that action -is not HA; it uses storage tied to the kubernetes node where the pod is running. -Consequently, if the registry pod is migrated from one node to another, you will -need to re-publish the images. - -#### Example usage - -Create the relevant authentication files. Let's say you want user ```userA``` -to authenticate with the password ```passwordA```. Then you'll do: - -``` -echo "userA:passwordA" > htpasswd-plain -htpasswd -c -b -B htpasswd userA passwordA -``` - -(the `htpasswd` program comes with the ```apache2-utils``` package) - -Assuming that your registry will be reachable at ```myregistry.company.com```, -you already have your TLS key in the ```registry.key``` file, and your TLS -certificate (with ```myregistry.company.com``` as Common Name) in the ```registry.crt``` file, you would then run: - -``` -juju run-action kubernetes-worker/0 registry domain=myregistry.company.com htpasswd="$(base64 -w0 htpasswd)" htpasswd-plain="$(base64 -w0 htpasswd-plain)" tlscert="$(base64 -w0 registry.crt)" tlskey="$(base64 -w0 registry.key)" ingress=true -``` - -If you then decide that you want to delete the registry, just run: - -``` -juju run-action kubernetes-worker/0 registry delete=true ingress=true -``` - - -{{% /capture %}} - - diff --git a/content/en/docs/getting-started-guides/ubuntu/rancher.md b/content/en/docs/getting-started-guides/ubuntu/rancher.md deleted file mode 100644 index dea8d44348..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/rancher.md +++ /dev/null @@ -1,360 +0,0 @@ ---- -title: Rancher Integration with Ubuntu Kubernetes -content_template: templates/task ---- - -{{% capture overview %}} -This repository explains how to deploy Rancher 2.0alpha on Canonical Kubernetes. - -These steps are currently in alpha/testing phase and will most likely change. - -The original documentation for this integration can be found at [https://github.com/CalvinHartwell/canonical-kubernetes-rancher/](https://github.com/CalvinHartwell/canonical-kubernetes-rancher/). - -{{% /capture %}} -{{% capture prerequisites %}} -To use this guide, you must have a working kubernetes cluster that was deployed using Canonical's juju. - -The full instructions for deploying Kubernetes with juju can be found at [/docs/getting-started-guides/ubuntu/installation/](/docs/getting-started-guides/ubuntu/installation/). -{{% /capture %}} - - -{{% capture steps %}} -## Deploying Rancher - -To deploy Rancher, we just need to run the Rancher container workload on-top of Kubernetes. Rancher provides their containers through dockerhub ([https://hub.docker.com/r/rancher/server/tags/](https://hub.docker.com/r/rancher/server/tags/)) and can be downloaded freely from the internet. - -If you're running your own registry or have an offline deployment, the container should be downloaded and pushed to a private registry before proceeding. - -### Deploying Rancher with a nodeport - -First create a yaml file which defines how to deploy Rancher on kubernetes. Save the file as cdk-rancher-nodeport.yaml: - -``` - --- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: cluster-admin -subjects: - - kind: ServiceAccount - name: default - namespace: default -roleRef: - kind: ClusterRole - name: cluster-admin - apiGroup: rbac.authorization.k8s.io ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: cluster-admin -rules: -- apiGroups: - - '*' - resources: - - '*' - verbs: - - '*' -- nonResourceURLs: - - '*' - verbs: - - '*' ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - creationTimestamp: null - labels: - app: rancher - name: rancher -spec: - replicas: 1 - selector: - matchLabels: - app: rancher - ima: pod - strategy: {} - template: - metadata: - creationTimestamp: null - labels: - app: rancher - ima: pod - spec: - containers: - - image: rancher/server:preview - imagePullPolicy: Always - name: rancher - ports: - - containerPort: 80 - - containerPort: 443 - livenessProbe: - httpGet: - path: / - port: 80 - initialDelaySeconds: 5 - timeoutSeconds: 30 - resources: {} - restartPolicy: Always - serviceAccountName: "" -status: {} ---- -apiVersion: v1 -kind: Service -metadata: - name: rancher - labels: - app: rancher -spec: - ports: - - port: 443 - protocol: TCP - targetPort: 443 - selector: - app: rancher ---- -apiVersion: v1 -kind: Service -metadata: - name: rancher-nodeport -spec: - type: NodePort - selector: - app: rancher - ports: - - name: rancher-api - protocol: TCP - nodePort: 30443 - port: 443 - targetPort: 443 -``` - -Once kubectl is running and working, run the following command to deploy Rancher: - -``` - kubectl apply -f cdk-rancher-nodeport.yaml -``` - -Now we need to open this nodeport so we can access it. For that, we can use juju. We need to run the open-port command for each of the worker nodes in our cluster. Inside the cdk-rancher-nodeport.yaml file, the nodeport has been set to 30443. Below shows how to open the port on each of the worker nodes: - -``` - # repeat this for each kubernetes worker in the cluster. - juju run --unit kubernetes-worker/0 "open-port 30443" - juju run --unit kubernetes-worker/1 "open-port 30443" - juju run --unit kubernetes-worker/2 "open-port 30443" -``` - -Rancher can now be accessed on this port through a worker IP or DNS entries if you have created them. It is generally recommended that you create a DNS entry for each of the worker nodes in your cluster. For example, if you have three worker nodes and you own the domain example.com, you could create three A records, one for each worker in the cluster. - -As creating DNS entries is outside of the scope of this document, we will use the freely available xip.io service which can return A records for an IP address which is part of the domain name. For example, if you have the domain rancher.35.178.130.245.xip.io, the xip.io service will automatically return the IP address 35.178.130.245 as an A record which is useful for testing purposes. For your deployment, the IP address 35.178.130.245 should be replaced with one of your worker IP address, which can be found using Juju or AWS: - -``` - calvinh@ubuntu-ws:~/Source/cdk-rancher$ juju status - -# ... output omitted. - -Unit Workload Agent Machine Public address Ports Message -easyrsa/0* active idle 0 35.178.118.232 Certificate Authority connected. -etcd/0* active idle 1 35.178.49.31 2379/tcp Healthy with 3 known peers -etcd/1 active idle 2 35.177.99.171 2379/tcp Healthy with 3 known peers -etcd/2 active idle 3 35.178.125.161 2379/tcp Healthy with 3 known peers -kubeapi-load-balancer/0* active idle 4 35.178.37.87 443/tcp Loadbalancer ready. -kubernetes-master/0* active idle 5 35.177.239.237 6443/tcp Kubernetes master running. - flannel/0* active idle 35.177.239.237 Flannel subnet 10.1.27.1/24 -kubernetes-worker/0* active idle 6 35.178.130.245 80/tcp,443/tcp,30443/tcp Kubernetes worker running. - flannel/2 active idle 35.178.130.245 Flannel subnet 10.1.82.1/24 -kubernetes-worker/1 active idle 7 35.178.121.29 80/tcp,443/tcp,30443/tcp Kubernetes worker running. - flannel/3 active idle 35.178.121.29 Flannel subnet 10.1.66.1/24 -kubernetes-worker/2 active idle 8 35.177.144.76 80/tcp,443/tcp,30443/tcp Kubernetes worker running. - flannel/1 active idle 35.177.144.76 - -# Note the IP addresses for the kubernetes-workers in the example above. You should pick one of the public addresses. -``` - -Try opening up Rancher in your browser using the nodeport and the domain name or ip address: - -``` - # replace the IP address with one of your Kubernetes worker, find this from juju status command. - wget https://35.178.130.245.xip.io:30443 --no-check-certificate - - # this should also work - wget https://35.178.130.245:30443 --no-check-certificate -``` - -If you need to make any changes to the kubernetes configuration file, edit the yaml file and then just use apply again: - -``` - kubectl apply -f cdk-rancher-nodeport.yaml -``` - -### Deploying Rancher with an ingress rule - -It is also possible to deploy Rancher using an ingress rule. This has the added benefit of not requiring additional ports to be opened up on the Kubernetes cluster. First create a yaml file to describe the deployment called cdk-rancher-ingress.yaml which should contain the following: - -``` ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: cluster-admin -subjects: - - kind: ServiceAccount - name: default - namespace: default -roleRef: - kind: ClusterRole - name: cluster-admin - apiGroup: rbac.authorization.k8s.io ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: cluster-admin -rules: -- apiGroups: - - '*' - resources: - - '*' - verbs: - - '*' -- nonResourceURLs: - - '*' - verbs: - - '*' ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - creationTimestamp: null - labels: - app: rancher - name: rancher -spec: - replicas: 1 - selector: - matchLabels: - app: rancher - strategy: {} - template: - metadata: - creationTimestamp: null - labels: - app: rancher - spec: - containers: - - image: rancher/server:preview - imagePullPolicy: Always - name: rancher - ports: - - containerPort: 443 - livenessProbe: - httpGet: - path: / - port: 80 - initialDelaySeconds: 5 - timeoutSeconds: 30 - resources: {} - restartPolicy: Always - serviceAccountName: "" -status: {} ---- -apiVersion: v1 -kind: Service -metadata: - name: rancher - labels: - app: rancher -spec: - ports: - - port: 443 - targetPort: 443 - protocol: TCP - selector: - app: rancher ---- -apiVersion: extensions/v1beta1 -kind: Ingress -metadata: - name: rancher - annotations: - kubernetes.io/tls-acme: "true" - ingress.kubernetes.io/secure-backends: "true" -spec: - tls: - - hosts: - - rancher.34.244.118.135.xip.io - rules: - - host: rancher.34.244.118.135.xip.io - http: - paths: - - path: / - backend: - serviceName: rancher - servicePort: 443 -``` - -It is generally recommended that you create a DNS entry for each of the worker nodes in your cluster. For example, if you have three worker nodes and you own the domain example.com, you could create three A records, one for each worker in the cluster. - -As creating DNS entries is outside of the scope of this tutorial, we will use the freely available xip.io service which can return A records for an IP address which is part of the domain name. For example, if you have the domain rancher.35.178.130.245.xip.io, the xip.io service will automatically return the IP address 35.178.130.245 as an A record which is useful for testing purposes. - -For your deployment, the IP address 35.178.130.245 should be replaced with one of your worker IP address, which can be found using Juju or AWS: - -``` - calvinh@ubuntu-ws:~/Source/cdk-rancher$ juju status - -# ... output omitted. - -Unit Workload Agent Machine Public address Ports Message -easyrsa/0* active idle 0 35.178.118.232 Certificate Authority connected. -etcd/0* active idle 1 35.178.49.31 2379/tcp Healthy with 3 known peers -etcd/1 active idle 2 35.177.99.171 2379/tcp Healthy with 3 known peers -etcd/2 active idle 3 35.178.125.161 2379/tcp Healthy with 3 known peers -kubeapi-load-balancer/0* active idle 4 35.178.37.87 443/tcp Loadbalancer ready. -kubernetes-master/0* active idle 5 35.177.239.237 6443/tcp Kubernetes master running. - flannel/0* active idle 35.177.239.237 Flannel subnet 10.1.27.1/24 -kubernetes-worker/0* active idle 6 35.178.130.245 80/tcp,443/tcp,30443/tcp Kubernetes worker running. - flannel/2 active idle 35.178.130.245 Flannel subnet 10.1.82.1/24 -kubernetes-worker/1 active idle 7 35.178.121.29 80/tcp,443/tcp,30443/tcp Kubernetes worker running. - flannel/3 active idle 35.178.121.29 Flannel subnet 10.1.66.1/24 -kubernetes-worker/2 active idle 8 35.177.144.76 80/tcp,443/tcp,30443/tcp Kubernetes worker running. - flannel/1 active idle 35.177.144.76 - -# Note the IP addresses for the kubernetes-workers in the example above. You should pick one of the public addresses. -``` - -Looking at the output from the juju status above, the Public Address (35.178.130.245) can be used to create a xip.io DNS entry (rancher.35.178.130.245.xip.io) which should be placed into the cdk-rancher-ingress.yaml file. You could also create your own DNS entry as long as it resolves to each of the worker nodes or one of them it will work fine: - -``` - # The xip.io domain should appear in two places in the file, change both entries. - cat cdk-rancher-ingress.yaml | grep xip.io - - host: rancher.35.178.130.245.xip.io -``` - -Once you've edited the ingress rule to reflect your DNS entries, run the kubectl apply -f cdk-rancher-ingress.yaml to deploy Kubernetes: - -``` - kubectl apply -f cdk-rancher-ingress.yaml -``` - -Rancher can now be accessed on the regular 443 through a worker IP or DNS entries if you have created them. Try opening it up in your browser: - -``` - # replace the IP address with one of your Kubernetes worker, find this from juju status command. - wget https://35.178.130.245.xip.io:443 --no-check-certificate -``` - -If you need to make any changes to the kubernetes configuration file, edit the yaml file and then just use apply again: - -``` - kubectl apply -f cdk-rancher-ingress.yaml -``` - -### Removing Rancher - -You can remove Rancher from your cluster using kubectl. Deleting constructs in Kubernetes is as simple as creating them: - -``` - # If you used the nodeport example change the yaml filename if you used the ingress example. - kubectl delete -f cdk-rancher-nodeport.yaml -``` -{{% /capture %}} diff --git a/content/en/docs/getting-started-guides/ubuntu/scaling.md b/content/en/docs/getting-started-guides/ubuntu/scaling.md deleted file mode 100644 index 45dd80044e..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/scaling.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: Scaling -content_template: templates/task ---- - -{{% capture overview %}} -This page shows how to horizontally scale master and worker nodes on a cluster. -{{% /capture %}} - -{{% capture prerequisites %}} -This page assumes you have a working Juju deployed cluster. - -Any of the applications can be scaled out post-deployment. The charms -update the status messages with progress, so it is recommended to run. - -``` -watch -c juju status --color -``` -{{% /capture %}} - -{{% capture steps %}} -## Kubernetes masters - -The provided Kubernetes master nodes act as a control plane for the cluster. -The deployment has been designed so that these nodes can be scaled independently -of worker nodes to allow for more operational flexibility. -To scale a master node up, simply execute: - - juju add-unit kubernetes-master - -This will add another master node to the control plane. -See the [building high-availability clusters](/docs/admin/high-availability) -section of the documentation for more information. - -## Kubernetes workers - -The kubernetes-worker nodes are the load-bearing units of a Kubernetes cluster. - -By default pods are automatically spread throughout the kubernetes-worker units -that you have deployed. - -To add more kubernetes-worker units to the cluster: - -``` -juju add-unit kubernetes-worker -``` - -or specify machine constraints to create larger nodes: - -``` -juju set-constraints kubernetes-worker "cpu-cores=8 mem=32G" -juju add-unit kubernetes-worker -``` - -Refer to the -[machine constraints documentation](https://jujucharms.com/docs/stable/charms-constraints) -for other machine constraints that might be useful for the kubernetes-worker units. - -## etcd - -Etcd is used as a key-value store for the Kubernetes cluster. The bundle -defaults to one instance in this cluster. - -For quorum reasons it is recommended to keep an odd number of etcd nodes. 3, 5, 7, and 9 nodes are the recommended amount of nodes, depending on your cluster size. The CoreOS etcd documentation has a chart for the -[optimal cluster size](https://coreos.com/etcd/docs/latest/admin_guide.html#optimal-cluster-size) -to determine fault tolerance. - -To add an etcd unit: - -``` -juju add-unit etcd -``` - -Shrinking of an etcd cluster after growth is not recommended. - -## Juju controller - -A single node is responsible for coordinating with all the Juju agents -on each machine that manage Kubernetes; it is called the controller node. -For production deployments it is recommended to enable HA of the controller node: - - juju enable-ha - -Enabling HA results in 3 controller nodes, this should be sufficient for most use cases. 5 and 7 controller nodes are also supported for extra large deployments. - -For more information, see "Controller high availability" topic in [Juju documentation](https://docs.jujucharms.com). -{{% /capture %}} diff --git a/content/en/docs/getting-started-guides/ubuntu/security.md b/content/en/docs/getting-started-guides/ubuntu/security.md deleted file mode 100644 index 89e70ea822..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/security.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: Security Considerations -content_template: templates/task ---- - -{{% capture overview %}} -By default all connections between every provided node are secured via TLS by easyrsa, including the etcd cluster. - -This page explains the security considerations of a deployed cluster and production recommendations. -{{% /capture %}} -{{% capture prerequisites %}} -This page assumes you have a working Juju deployed cluster. -{{% /capture %}} - - -{{% capture steps %}} -## Implementation - -The TLS and easyrsa implementations use the following [layers](https://jujucharms.com/docs/2.2/developer-layers). - -[layer-tls-client](https://github.com/juju-solutions/layer-tls-client) -[layer-easyrsa](https://github.com/juju-solutions/layer-easyrsa) - - -## Limiting ssh access - -By default the administrator can ssh to any deployed node in a cluster. You can mass disable ssh access to the cluster nodes by issuing the following command. - - juju model-config proxy-ssh=true - -Note: The Juju controller node will still have open ssh access in your cloud, and will be used as a jump host in this case. - -Refer to the [model management](https://jujucharms.com/docs/2.2/models) page in the Juju documentation for instructions on how to manage ssh keys. -{{% /capture %}} - - diff --git a/content/en/docs/getting-started-guides/ubuntu/storage.md b/content/en/docs/getting-started-guides/ubuntu/storage.md deleted file mode 100644 index 8a93e53a45..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/storage.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -title: Storage -content_template: templates/task ---- - -{{% capture overview %}} -This page explains how to install and configure persistent storage on a cluster. -{{% /capture %}} -{{% capture prerequisites %}} -This page assumes you have a working Juju deployed cluster. -{{% /capture %}} - -{{% capture steps %}} -## Ceph Persistent Volumes - -The Canonical Distribution of Kubernetes allows you to connect with durable -storage devices such as [Ceph](http://ceph.com). When paired with the -[Juju Storage](https://jujucharms.com/docs/2.0/charms-storage) feature you -can add durable storage easily and across clouds. - -Deploy a minimum of three ceph-mon and three ceph-osd units. - -``` -juju deploy cs:ceph-mon -n 3 -juju deploy cs:ceph-osd -n 3 -``` - -Relate the units together: -``` -juju add-relation ceph-mon ceph-osd -``` - -List the storage pools available to Juju for your cloud: - - juju storage-pools - -Output: -``` -Name Provider Attrs -ebs ebs -ebs-ssd ebs volume-type=ssd -loop loop -rootfs rootfs -tmpfs tmpfs -``` - -{{< note >}} -This listing is for the Amazon Web Services public cloud. Different clouds may have -different pool names. -{{< /note >}} - -Add a storage pool to the ceph-osd charm by NAME,SIZE,COUNT: - -``` -juju add-storage ceph-osd/0 osd-devices=ebs,10G,1 -juju add-storage ceph-osd/1 osd-devices=ebs,10G,1 -juju add-storage ceph-osd/2 osd-devices=ebs,10G,1 -``` - -Next relate the storage cluster with the Kubernetes cluster: - -``` -juju add-relation kubernetes-master ceph-mon -``` - -We are now ready to enlist -[Persistent Volumes](/docs/concepts/storage/persistent-volumes/) -in Kubernetes which our workloads can consume via Persistent Volume (PV) claims. - -``` -juju run-action kubernetes-master/0 create-rbd-pv name=test size=50 -``` - -This example created a "test" Rados Block Device (rbd) in the size of 50 MB. - -Use watch on your Kubernetes cluster like the following, you should see the PV -become enlisted and be marked as available: - - watch kubectl get pv - -Output: - -``` -NAME CAPACITY ACCESSMODES STATUS CLAIM REASON AGE - -test 50M RWO Available 10s -``` - -To consume these Persistent Volumes, your pods will need an associated -Persistent Volume Claim with them, and is outside the scope of this README. See the -[Persistent Volumes](/docs/concepts/storage/persistent-volumes/) -documentation for more information. -{{% /capture %}} - - diff --git a/content/en/docs/getting-started-guides/ubuntu/troubleshooting.md b/content/en/docs/getting-started-guides/ubuntu/troubleshooting.md deleted file mode 100644 index 9c40c49fef..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/troubleshooting.md +++ /dev/null @@ -1,179 +0,0 @@ ---- -title: Troubleshooting -content_template: templates/task ---- - -{{% capture overview %}} -This document with highlighting how to troubleshoot the deployment of a Kubernetes cluster, it will not cover debugging of workloads inside Kubernetes. -{{% /capture %}} -{{% capture prerequisites %}} -This page assumes you have a working Juju deployed cluster. -{{% /capture %}} - -{{% capture steps %}} -## Understanding Cluster Status - -Using `juju status` can give you some insight as to what's happening in a cluster: - - -``` -Model Controller Cloud/Region Version -kubes work-multi aws/us-east-2 2.0.2.1 - -App Version Status Scale Charm Store Rev OS Notes -easyrsa 3.0.1 active 1 easyrsa jujucharms 3 ubuntu -etcd 2.2.5 active 1 etcd jujucharms 17 ubuntu -flannel 0.6.1 active 2 flannel jujucharms 6 ubuntu -kubernetes-master 1.4.5 active 1 kubernetes-master jujucharms 8 ubuntu exposed -kubernetes-worker 1.4.5 active 1 kubernetes-worker jujucharms 11 ubuntu exposed - -Unit Workload Agent Machine Public address Ports Message -easyrsa/0* active idle 0/lxd/0 10.0.0.55 Certificate Authority connected. -etcd/0* active idle 0 52.15.47.228 2379/tcp Healthy with 1 known peers. -kubernetes-master/0* active idle 0 52.15.47.228 6443/tcp Kubernetes master services ready. - flannel/1 active idle 52.15.47.228 Flannel subnet 10.1.75.1/24 -kubernetes-worker/0* active idle 1 52.15.177.233 80/tcp,443/tcp Kubernetes worker running. - flannel/0* active idle 52.15.177.233 Flannel subnet 10.1.63.1/24 - -Machine State DNS Inst id Series AZ -0 started 52.15.47.228 i-0bb211a18be691473 xenial us-east-2a -0/lxd/0 started 10.0.0.55 juju-153b74-0-lxd-0 xenial -1 started 52.15.177.233 i-0502d7de733be31bb xenial us-east-2b -``` - -In this example we can glean some information. The `Workload` column will show the status of a given service. The `Message` section will show you the health of a given service in the cluster. During deployment and maintenance these workload statuses will update to reflect what a given node is doing. For example the workload my say `maintenance` while message will describe this maintenance as `Installing docker`. - -During normal operation the Workload should read `active`, the Agent column (which reflects what the Juju agent is doing) should read `idle`, and the messages will either say `Ready` or another descriptive term. `juju status --color` will also return all green results when a cluster's deployment is healthy. - -Status can become unwieldy for large clusters, it is then recommended to check status on individual services, for example to check the status on the workers only: - - juju status kubernetes-worker - -or just on the etcd cluster: - - juju status etcd - -Errors will have an obvious message, and will return a red result when used with `juju status --color`. Nodes that come up in this manner should be investigated. - -## SSHing to units - -You can ssh to individual units easily with the following convention, `juju ssh /`: - - juju ssh kubernetes-worker/3 - -Will automatically ssh you to the 3rd worker unit. - - juju ssh easyrsa/0 - -This will automatically ssh you to the easyrsa unit. - -## Collecting debug information - -Sometimes it is useful to collect all the information from a cluster to share with a developer to identify problems. This is best accomplished with [CDK Field Agent](https://github.com/juju-solutions/cdk-field-agent). - -Download and execute the collect.py script from [CDK Field Agent](https://github.com/juju-solutions/cdk-field-agent) on a box that has a Juju client configured with the current controller and model pointing at the CDK deployment of interest. - -Running the script will generate a tarball of system information and includes basic information such as systemctl status, Juju logs, charm unit data, etc. Additional application-specific information may be included as well. - -## Common Problems - -### Load Balancer interfering with Helm - -This section assumes you have a working deployment of Kubernetes via Juju using a Load Balancer for the API, and that you are using Helm to deploy charts. - -To deploy Helm you will have run: - -``` -helm init -$HELM_HOME has been configured at /home/ubuntu/.helm -Tiller (the helm server side component) has been installed into your Kubernetes Cluster. -Happy Helming! -``` - -Then when using helm you may see one of the following errors: - -* Helm doesn't get the version from the Tiller server - -``` -helm version -Client: &version.Version{SemVer:"v2.1.3", GitCommit:"5cbc48fb305ca4bf68c26eb8d2a7eb363227e973", GitTreeState:"clean"} -Error: cannot connect to Tiller -``` - -* Helm cannot install your chart - -``` -helm install --debug -Error: forwarding ports: error upgrading connection: Upgrade request required -``` - -This is caused by the API load balancer not forwarding ports in the context of the helm client-server relationship. To deploy using helm, you will need to follow these steps: - -1. Expose the Kubernetes Master service - - ``` - juju expose kubernetes-master - ``` - -1. Identify the public IP address of one of your masters - - ``` - juju status kubernetes-master - Model Controller Cloud/Region Version - production k8s-admin aws/us-east-1 2.0.0 - - App Version Status Scale Charm Store Rev OS Notes - flannel 0.6.1 active 1 flannel jujucharms 7 ubuntu - kubernetes-master 1.5.1 active 1 kubernetes-master jujucharms 10 ubuntu exposed - - Unit Workload Agent Machine Public address Ports Message - kubernetes-master/0* active idle 5 54.210.100.102 6443/tcp Kubernetes master running. - flannel/0 active idle 54.210.100.102 Flannel subnet 10.1.50.1/24 - - Machine State DNS Inst id Series AZ - 5 started 54.210.100.102 i-002b7150639eb183b xenial us-east-1a - - Relation Provides Consumes Type - certificates easyrsa kubernetes-master regular - etcd etcd flannel regular - etcd etcd kubernetes-master regular - cni flannel kubernetes-master regular - loadbalancer kubeapi-load-balancer kubernetes-master regular - cni kubernetes-master flannel subordinate - cluster-dns kubernetes-master kubernetes-worker regular - cni kubernetes-worker flannel subordinate - ``` - - In this context the public IP address is 54.210.100.102. - - If you want to access this data programmatically you can use the JSON output: - - ``` - juju show-status kubernetes-master --format json | jq --raw-output '.applications."kubernetes-master".units | keys[]' - 54.210.100.102 - ``` - -1. Update the kubeconfig file - - Identify the kubeconfig file or section used for this cluster, and edit the server configuration. - - By default, it will look like ```https://54.213.123.123:443```. Replace it with the Kubernetes Master endpoint ```https://54.210.100.102:6443``` and save. - - Note that the default port used by CDK for the Kubernetes Master API is 6443 while the port exposed by the load balancer is 443. - -1. Start helm again! - - ``` - helm install --debug - Created tunnel using local port: '36749' - SERVER: "localhost:36749" - CHART PATH: /home/ubuntu/.helm/ - NAME: - ... - ... - ``` - -## Logging and monitoring - -By default there is no log aggregation of the Kubernetes nodes, each node logs locally. Please read over the [logging](/docs/getting-started-guides/ubuntu/logging/) page for more information. -{{% /capture %}} diff --git a/content/en/docs/getting-started-guides/ubuntu/upgrades.md b/content/en/docs/getting-started-guides/ubuntu/upgrades.md deleted file mode 100644 index 5f87c0b3c8..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/upgrades.md +++ /dev/null @@ -1,164 +0,0 @@ ---- -title: Upgrades -content_template: templates/task ---- - -{{% capture overview %}} -This page will outline how to manage and execute a Kubernetes upgrade. -{{% /capture %}} - -{{% capture prerequisites %}} -This page assumes you have a working deployed cluster. - -{{< warning >}} -You should always back up all your data before attempting an upgrade. -Don't forget to include the workload inside your cluster! -Refer to the [backup documentation](/docs/getting-started-guides/ubuntu/backups). -{{< /warning >}} - -{{% /capture %}} - -{{% capture steps %}} -## Patch kubernetes upgrades for example 1.9.0 -> 1.9.1 - -Clusters are transparently upgraded to the latest Kubernetes patch release. -To be clear, a cluster deployed using the 1.9/stable channel -will transparently receive unattended upgrades for the 1.9.X Kubernetes -releases. -The upgrade causes no disruption to the operation of the cluster and requires -no intervention from a cluster administrator. -Each patch release is evaluated by the -Canonical Kubernetes Distribution team. -Once a patch release passes internal testing and is deemed safe for upgrade, -it is packaged in snap format and pushed to the stable channel. - - -## Upgrading a minor Kubernetes release for example 1.8.1 -> 1.9.0 - - -The Kubernetes charms follow the Kubernetes releases. Please consult -your support plan on the upgrade frequency. Important operational considerations -and changes in behaviour will always be documented in the release notes. - -### Upgrade etcd - -Backing up etcd requires an export and snapshot, refer to the -[backup documentation](/docs/getting-started-guides/ubuntu/backups) to create a snapshot. -After the snapshot, upgrade the etcd service with: - - juju upgrade-charm etcd - -This will handle upgrades between minor versions of etcd. Instructions on how to upgrade from 2.x to 3.x can be found [here](https://github.com/juju-solutions/bundle-canonical-kubernetes/wiki/Etcd-2.3-to-3.x-upgrade) in the juju-solutions wiki. - -### Upgrade kubeapi-load-balancer - -The Kubernetes Charms are generally all updated and released at the same time. A core part of a cluster on Ubuntu is the kubeapi-load-balancer component. Incorrect or missing changes there can have an effect on API availability and access controls. To ensure API service continuity for the master and workers when they are updated, this upgrade needs to precede them. - -To upgrade the charm run: - - juju upgrade-charm kubeapi-load-balancer - -### Upgrade Kubernetes - -The Kubernetes Charms use snap channels to drive payloads. -The channels are defined by `X.Y/channel` where `X.Y` is the `major.minor` release -of Kubernetes (for example 1.9) and `channel` is one of the four following channels: - -| Channel name | Description | -| ------------------- | ------------ | -| stable | The latest stable released patch version of Kubernetes | -| candidate | Release candidate releases of Kubernetes | -| beta | Latest alpha or beta of Kubernetes for that minor release | -| edge | Nightly builds of that minor release of Kubernetes | - -If a release isn't available, the next highest channel is used. -For example, 1.9/beta will load `/candidate` or `/stable` depending on availability of release. -Development versions of Kubernetes are available in the edge channel for each minor release. -There is no guarantee that edge snaps will work with the current charms. - -### Master Upgrades - -First you need to upgrade the masters: - - juju upgrade-charm kubernetes-master - -{{< note >}} -Always upgrade the masters before the workers. -{{< /note >}} - -Once the latest charm is deployed, the channel for Kubernetes can be selected by issuing the following: - - juju config kubernetes-master channel=1.x/stable - -Where `x` is the minor version of Kubernetes. For example, `1.9/stable`. See above for Channel definitions. -Once you've configured kubernetes-master with the appropriate channel, run the upgrade action on each master: - - juju run-action kubernetes-master/0 upgrade - juju run-action kubernetes-master/1 upgrade - ... - -### Worker Upgrades - -Two methods of upgrading workers are supported. -[Blue/Green Deployment](http://martinfowler.com/bliki/BlueGreenDeployment.html) -and upgrade-in-place. Both methods are provided for operational flexibility and both -are supported and tested. Blue/Green will require more hardware up front than in-place, -but is a safer upgrade route. - -#### Blue/green worker upgrade - -Given a deployment where the workers are named kubernetes-blue. - -Deploy new workers: - - juju deploy kubernetes-green - -Pause the old workers so your workload migrates: - - juju run-action kubernetes-blue/# pause - -Verify old workloads have migrated with: - - kubectl get pod -o wide - -Tear down old workers with: - - juju remove-application kubernetes-blue - -#### In place worker upgrade - - juju upgrade-charm kubernetes-worker - juju config kubernetes-worker channel=1.x/stable - -Where `x` is the minor version of Kubernetes. For example, `1.9/stable`. -See above for Channel definitions. Once you've configured kubernetes-worker with the appropriate channel, -run the upgrade action on each worker: - - juju run-action kubernetes-worker/0 upgrade - juju run-action kubernetes-worker/1 upgrade - ... - -### Verify upgrade - -`kubectl version` should return the newer version. - -It is recommended to rerun a [cluster validation](/docs/getting-started-guides/ubuntu/validation) -to ensure that the cluster upgrade has successfully completed. - -### Upgrade Flannel - -Upgrading flannel can be done at any time, it is independent of Kubernetes upgrades. -Be advised that networking is interrupted during the upgrade. You can initiate a flannel upgrade with: - - juju upgrade-charm flannel - -### Upgrade easyrsa - -Upgrading easyrsa can be done at any time, it is independent of Kubernetes upgrades. -Upgrading easyrsa should result in zero downtime as it is not a running service: - - juju upgrade-charm easyrsa - -{{% /capture %}} - - diff --git a/content/en/docs/getting-started-guides/ubuntu/validation.md b/content/en/docs/getting-started-guides/ubuntu/validation.md deleted file mode 100644 index c61c1ec4be..0000000000 --- a/content/en/docs/getting-started-guides/ubuntu/validation.md +++ /dev/null @@ -1,169 +0,0 @@ ---- -title: Validation - End-to-end Testing -content_template: templates/task ---- - -{{% capture overview %}} -This page will outline how to ensure that a Juju-deployed Kubernetes -cluster has stood up correctly and is ready to accept workloads. -{{% /capture %}} - -{{% capture prerequisites %}} -This page assumes you have a working Juju deployed cluster. -{{% /capture %}} - -{{% capture steps %}} -## End-to-end testing - -End-to-end (e2e) tests for Kubernetes provide a mechanism to test end-to-end -behavior of the system, and is the last signal to ensure end user operations -match developer specifications. Although unit and integration tests provide a -good signal, in a distributed system like Kubernetes it is not uncommon that a -minor change may pass all unit and integration tests, but cause unforeseen -changes at the system level. - -The primary objectives of the e2e tests are to ensure a consistent and reliable -behavior of the kubernetes code base, and to catch hard-to-test bugs before -users do, when unit and integration tests are insufficient. - -End-to-end tests will pass on a properly running CDK cluster outside of bugs in the tests. - -### Deploy kubernetes-e2e charm - -To deploy the end-to-end test suite, you need to relate the `kubernetes-e2e` charm -to your existing kubernetes-master nodes and easyrsa: - -``` -juju deploy cs:~containers/kubernetes-e2e -juju add-relation kubernetes-e2e easyrsa -juju add-relation kubernetes-e2e:kubernetes-master kubernetes-master:kube-api-endpoint -juju add-relation kubernetes-e2e:kube-control kubernetes-master:kube-control -``` - -Once the relations have settled, you can do `juju status` until the workload status results in - `Ready to test.` - you may then kick off an end to end validation test. - -### Running the e2e test - -The e2e test is encapsulated as an action to ensure consistent runs of the -end to end test. The defaults are sensible for most deployments. - - juju run-action kubernetes-e2e/0 test - - -### Tuning the e2e test - -The e2e test is configurable. By default it will focus on or skip the declared -conformance tests in a cloud agnostic way. Default behaviors are configurable. -This allows the operator to test only a subset of the conformance tests, or to -test more behaviors not enabled by default. You can see all tunable options on -the charm by inspecting the schema output of the actions: - - juju actions kubernetes-e2e --format=yaml --schema - -Output: - -``` -test: - description: Run end-to-end validation test suite - properties: - focus: - default: \[Conformance\] - description: Regex focus for executing the test - type: string - skip: - default: \[Flaky\] - description: Regex of tests to skip - type: string - timeout: - default: 30000 - description: Timeout in nanoseconds - type: integer - title: test - type: object -``` - -As an example, you can run a more limited set of tests for rapid validation of -a deployed cluster. The following example will skip the `Flaky`, `Slow`, and -`Feature` labeled tests: - - juju run-action kubernetes-e2e/0 test skip='\[(Flaky|Slow|Feature:.*)\]' - -{{< note >}} -The escaping of the regex due to how bash handles brackets. -{{< /note >}} - -To see the different types of tests the Kubernetes end-to-end charm has access -to, we encourage you to see the [upstream documentation on the different types -of tests](https://git.k8s.io/community/contributors/devel/e2e-tests.md#kinds-of-tests), -and to thoroughly understand what subsets of the tests you are running. - -### More information on end-to-end testing - -Along with the above descriptions, end-to-end testing is a much larger subject -than this readme can encapsulate. There is far more information in the -[end-to-end testing guide](https://git.k8s.io/community/contributors/devel/e2e-tests.md). - -### Evaluating end-to-end results - -It is not enough to just simply run the test. Result output is stored in two -places. The raw output of the e2e run is available in the `juju show-action-output` -command, as well as a flat file on disk on the `kubernetes-e2e` unit that -executed the test. - -{{< note >}} -The results will only be available once the action has -completed the test run. End-to-end testing can be quite time consuming, often -taking more than 1 hour, depending on configuration. -{{< /note >}} - -##### Accessing the results in a flat file - -Here's how to copy the output out as a file: - - juju run-action kubernetes-e2e/0 test - -Output: - - Action queued with id: 4ceed33a-d96d-465a-8f31-20d63442e51b - -Copy output to your local machine: - - juju scp kubernetes-e2e/0:4ceed33a-d96d-465a-8f31-20d63442e51b.log . - -##### Action result output - -Or you can just show the output inline: - - juju run-action kubernetes-e2e/0 test - -Output: - - Action queued with id: 4ceed33a-d96d-465a-8f31-20d63442e51b - -Show the results in your terminal: - - juju show-action-output 4ceed33a-d96d-465a-8f31-20d63442e51b - - -### Known issues - -The e2e test suite assumes egress network access. It will pull container -images from `gcr.io`. You will need to have this registry unblocked in your -firewall to successfully run e2e test results. Or you may use the exposed -proxy settings [properly configured](https://github.com/juju-solutions/bundle-canonical-kubernetes#proxy-configuration) -on the kubernetes-worker units. - -## Upgrading the e2e tests - -The e2e tests are always expanding; you can see if there's an upgrade -available by running `juju status kubernetes-e2e`. - -When an upgrade is available, upgrade your deployment: - - juju upgrade-charm kubernetes-e2e - -{{% /capture %}} - - - diff --git a/content/en/docs/setup/pick-right-solution.md b/content/en/docs/setup/pick-right-solution.md index 16ad1ae2cc..87d8ce6429 100644 --- a/content/en/docs/setup/pick-right-solution.md +++ b/content/en/docs/setup/pick-right-solution.md @@ -69,7 +69,7 @@ cluster. * [IBM Cloud Private-CE (Community Edition) on Linux Containers](https://github.com/HSBawa/icp-ce-on-linux-containers) is a Terraform/Packer/BASH based Infrastructure as Code (IaC) scripts to create a seven node (1 Boot, 1 Master, 1 Management, 1 Proxy and 3 Workers) LXD cluster on Linux Host. -* [Ubuntu on LXD](/docs/getting-started-guides/ubuntu/local/) supports a nine-instance deployment on localhost. +* [CDK on LXD](https://www.ubuntu.com/kubernetes/docs/install-local) supports a nine-instance deployment on localhost with LXD containers. ## Hosted Solutions @@ -128,7 +128,7 @@ few commands. These solutions are actively developed and have active community s * [AWS](/docs/setup/turnkey/aws/) * [Azure](/docs/setup/turnkey/azure/) * [CenturyLink Cloud](/docs/setup/turnkey/clc/) -* [Conjure-up Kubernetes with Ubuntu on AWS, Azure, Google Cloud, Oracle Cloud](/docs/getting-started-guides/ubuntu/) +* [Conjure-up Kubernetes with Ubuntu on AWS, Azure, Google Cloud, Oracle Cloud](https://www.ubuntu.com/kubernetes/docs/quickstart) * [Containership](https://containership.io/containership-platform) * [Docker Enterprise](https://www.docker.com/products/docker-enterprise) * [Gardener](https://gardener.cloud/) @@ -195,7 +195,7 @@ These solutions are combinations of cloud providers and operating systems not co * [Cloud Foundry Container Runtime (CFCR)](https://docs-cfcr.cfapps.io/) * [Gardener](https://gardener.cloud/) * [Kublr](www.kublr.com/kubernetes.io/setup-hosted-solution) -* [Kubernetes on Ubuntu](/docs/getting-started-guides/ubuntu/) +* [Kubernetes on Ubuntu](https://www.ubuntu.com/kubernetes/docs/quickstart) * [Kubespray](/docs/setup/custom-cloud/kubespray/) * [Rancher Kubernetes Engine (RKE)](https://github.com/rancher/rke) * [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-PKS) @@ -211,7 +211,7 @@ These solutions are combinations of cloud providers and operating systems not co * [Platform9 Managed Kubernetes as a Service](https://platform9.com/managed-kubernetes/) works on any infrastructure: on-premises, bare metal, or private/hybrid cloud. * [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-PKS) * [VMware vSphere](https://github.com/kubernetes/cloud-provider-vsphere) -* [VMware vSphere, OpenStack, or Bare Metal](/docs/getting-started-guides/ubuntu/) (uses Juju, Ubuntu and flannel) +* [VMware vSphere, OpenStack, or Bare Metal](https://www.ubuntu.com/kubernetes/docs/quickstart) (uses Juju, Ubuntu and flannel) ### Bare Metal @@ -219,7 +219,7 @@ These solutions are combinations of cloud providers and operating systems not co * [Docker Enterprise](https://www.docker.com/products/docker-enterprise) * [Fedora (Single Node)](/docs/getting-started-guides/fedora/fedora_manual_config/) * [Fedora (Multi Node)](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) -* [Kubernetes on Ubuntu](/docs/getting-started-guides/ubuntu/) +* [Kubernetes on Ubuntu](https://www.ubuntu.com/kubernetes/docs/quickstart) * [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) (OCP) Kubernetes platform by [Red Hat](https://www.redhat.com) * [Platform9 Managed Kubernetes as a Service](https://platform9.com/managed-kubernetes/) works on any infrastructure: on-premises, bare metal, or private/hybrid cloud. * [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-PKS) @@ -249,22 +249,22 @@ AppsCode.com | Saltstack | Debian | multi-support | [docs](https://ap AWS | CoreOS | CoreOS | flannel | [docs](/docs/setup/turnkey/aws/) | Community AWS | Saltstack | Debian | AWS | [docs](/docs/setup/turnkey/aws/) | Community ([@justinsb](https://github.com/justinsb)) AWS | kops | Debian | AWS | [docs](https://github.com/kubernetes/kops/) | Community ([@justinsb](https://github.com/justinsb)) -AWS | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) -Azure | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) +AWS | Juju | Ubuntu | flannel/calico/canal | [docs](https://www.ubuntu.com/kubernetes/docs/quickstart) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) +Azure | Juju | Ubuntu | flannel/calico/canal | [docs](https://www.ubuntu.com/kubernetes/docs/quickstart) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) Azure (IaaS) | | Ubuntu | Azure | [docs](/docs/setup/turnkey/azure/) | [Community (Microsoft)](https://github.com/Azure/acs-engine) Azure Kubernetes Service | | Ubuntu | Azure | [docs](https://docs.microsoft.com/en-us/azure/aks/) | Commercial Bare-metal | custom | CentOS | flannel | [docs](/docs/getting-started-guides/centos/centos_manual_config/) | Community ([@coolsvap](https://github.com/coolsvap)) Bare-metal | custom | Fedora | _none_ | [docs](/docs/getting-started-guides/fedora/fedora_manual_config/) | Project Bare-metal | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal)) -Bare Metal | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) -Bare-metal | custom | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu/) | Community ([@resouer](https://github.com/resouer), [@WIZARD-CXY](https://github.com/WIZARD-CXY)) +Bare Metal | Juju | Ubuntu | flannel/calico/canal | [docs](https://www.ubuntu.com/kubernetes/docs/quickstart) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) +Bare-metal | custom | Ubuntu | flannel | [docs](https://www.ubuntu.com/kubernetes/docs/quickstart) | Community ([@resouer](https://github.com/resouer), [@WIZARD-CXY](https://github.com/WIZARD-CXY)) CloudStack | Ansible | CoreOS | flannel | [docs](/docs/getting-started-guides/cloudstack/) | Community ([@sebgoa](https://github.com/sebgoa)) DCOS | Marathon | CoreOS/Alpine | custom | [docs](/docs/getting-started-guides/dcos/) | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md)) Digital Rebar | kubeadm | any | metal | [docs](/docs/setup/on-premises-metal/krib/) | Community ([@digitalrebar](https://github.com/digitalrebar)) Docker Enterprise | custom | [multi-support](https://success.docker.com/article/compatibility-matrix) | [multi-support](https://docs.docker.com/ee/ucp/kubernetes/install-cni-plugin/) | [docs](https://docs.docker.com/ee/) | Commercial Giant Swarm | | CoreOS | flannel and/or Calico | [docs](https://docs.giantswarm.io/) | Commercial GCE | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/) | Community ([@pires](https://github.com/pires)) -GCE | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) +GCE | Juju | Ubuntu | flannel/calico/canal | [docs](https://www.ubuntu.com/kubernetes/docs/quickstart) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) GCE | Saltstack | Debian | GCE | [docs](/docs/setup/turnkey/gce/) | Project Google Kubernetes Engine | | | GCE | [docs](https://cloud.google.com/kubernetes-engine/docs/) | Commercial IBM Cloud Kubernetes Service | | Ubuntu | IBM Cloud Networking + Calico | [docs](https://cloud.ibm.com/docs/containers?topic=containers-container_index#container_index) | Commercial @@ -274,10 +274,10 @@ Kublr | custom | multi-support | multi-support | [docs](htt Kubermatic | | multi-support | multi-support | [docs](http://docs.kubermatic.io/) | Commercial KVM | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal)) libvirt | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal)) -lxd | Juju | Ubuntu | flannel/canal | [docs](/docs/getting-started-guides/ubuntu/local/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) +lxd | Juju | Ubuntu | flannel/canal | [docs](https://www.ubuntu.com/kubernetes/docs/quickstart) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) Madcore.Ai | Jenkins DSL | Ubuntu | flannel | [docs](https://madcore.ai) | Community ([@madcore-ai](https://github.com/madcore-ai)) Mirantis Cloud Platform | Salt | Ubuntu | multi-support | [docs](https://docs.mirantis.com/mcp/) | Commercial -Oracle Cloud Infrastructure | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) +Oracle Cloud Infrastructure | Juju | Ubuntu | flannel/calico/canal | [docs](https://www.ubuntu.com/kubernetes/docs/quickstart) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) Oracle Cloud Infrastructure Container Engine for Kubernetes (OKE) | | | multi-support | [docs](https://docs.cloud.oracle.com/iaas/Content/ContEng/Concepts/contengoverview.htm) | Commercial oVirt | | | | [docs](/docs/setup/on-premises-vm/ovirt/) | Community ([@simon3z](https://github.com/simon3z)) Platform9 | | multi-support | multi-support | [docs](https://platform9.com/managed-kubernetes/) | Commercial @@ -286,7 +286,7 @@ Red Hat OpenShift | Ansible & CoreOS | RHEL & CoreOS | [multi-support](http Stackpoint.io | | multi-support | multi-support | [docs](https://stackpoint.io/) | Commercial Vagrant | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/) | Community ([@pires](https://github.com/pires), [@AntonioMeireles](https://github.com/AntonioMeireles)) VMware vSphere | any | multi-support | multi-support | [docs](https://github.com/kubernetes/cloud-provider-vsphere/tree/master/docs) | [Community](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/contactus.html) -VMware vSphere | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) +VMware vSphere | Juju | Ubuntu | flannel/calico/canal | [docs](https://www.ubuntu.com/kubernetes/docs/quickstart) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes) VMware Cloud PKS | | Photon OS | Canal | [docs](https://docs.vmware.com/en/VMware-Kubernetes-Engine/index.html) | Commercial VMware Enterprise PKS | BOSH | Ubuntu | VMware NSX-T/flannel | [docs](https://docs.vmware.com/en/VMware-Enterprise-PKS/) | Commercial VMware Essential PKS | any | multi-support | multi-support | [docs](https://cloud.vmware.com/vmware-essential-PKS) | Commercial