Merge branch 'master' into patch-1

This commit is contained in:
Alexander
2017-02-01 10:50:10 +00:00
committed by GitHub
8 changed files with 301 additions and 3 deletions
+1
View File
@@ -164,6 +164,7 @@ toc:
- docs/getting-started-guides/ubuntu/troubleshooting.md
- docs/getting-started-guides/ubuntu/decommissioning.md
- docs/getting-started-guides/ubuntu/calico.md
- docs/getting-started-guides/ubuntu/operational-considerations.md
- docs/getting-started-guides/ubuntu/glossary.md
- docs/getting-started-guides/ubuntu/local.md
- docs/getting-started-guides/ubuntu/logging.md
@@ -43,6 +43,7 @@ These are more in-depth guides for users choosing to run Kubernetes in productio
- [Storage](/docs/getting-started-guides/ubuntu/storage)
- [Troubleshooting](/docs/getting-started-guides/ubuntu/troubleshooting)
- [Decommissioning](/docs/getting-started-guides/ubuntu/decommissioning)
- [Operational Considerations](/docs/getting-started-guides/ubuntu/operational-considerations)
- [Glossary](/docs/getting-started-guides/ubuntu/glossary)
## Developer Guides
@@ -18,3 +18,43 @@ The `juju debug-log` will show all of the consolidated logs of all the Juju agen
See the [Juju documentation](https://jujucharms.com/docs/stable/troubleshooting-logs) for more information.
## Managing log verbosity
Log verbosity in Juju is set at the model level. You can adjust it at any time:
```
juju add-model k8s-development --config logging-config='<root>=DEBUG;unit=DEBUG'
```
and later
```
juju config-model k8s-production --config logging-config='<root>=ERROR;unit=ERROR'
```
In addition, the jujud daemon is started in debug mode by default on all controllers. To remove that behavior edit ```/var/lib/juju/init/jujud-machine-0/exec-start.sh``` on the controller node and comment the ```--debug``` section.
It then contains:
```
#!/usr/bin/env bash
# Set up logging.
touch '/var/log/juju/machine-0.log'
chown syslog:syslog '/var/log/juju/machine-0.log'
chmod 0600 '/var/log/juju/machine-0.log'
exec >> '/var/log/juju/machine-0.log'
exec 2>&1
# Run the script.
'/var/lib/juju/tools/machine-0/jujud' machine --data-dir '/var/lib/juju' --machine-id 0 # --debug
```
Then restart the service with:
```
sudo systemctl restart jujud-machine-0.service
```
See the [official documentation](https://jujucharms.com/docs/stable/models-config) for more information about logging and other model settings in Juju.
@@ -0,0 +1,160 @@
---
title: Operational Considerations
---
{% capture overview %}
This page gives recommendations and hints for people managing long lived clusters
{% endcapture %}
{% capture prerequisites %}
This page assumes you understand the basics of Juju and Kubernetes.
{% endcapture %}
{% capture steps %}
## Managing Juju
### Sizing your controller node
The Juju Controller:
* requires about 2 to 2.5GB RAM to operate.
* uses a MongoDB database as a storage backend for the configuration and state of the cluster. This database can grow significantly, and can also be the biggest consumer of CPU cycles on the instance
* aggregates and stores the log data of all services and units. Therefore, significant storage is needed for long lived models. If your intention is to keep the cluster running, make sure to provision at least 64GB for the logs.
To bootstrap a controller with constraints run the following command:
```
juju bootstrap --contraints "mem=8GB cpu-cores=4 root-disk=128G"
```
Juju will select the cheapest instance type matching your constraints on your target cloud. You can also use the ```instance-type``` constraint in conjunction with ```root-disk``` for strict control. For more information about the constraints available, refer to the [official documentation](https://jujucharms.com/docs/stable/reference-constraints)
Additional information about logging can be found in the [logging section](/docs/getting-started-guides/ubuntu/logging)
### SSHing into the Controller Node
By default, Juju will create a pair of SSH keys that it will use to automate the connection to units. They are stored on the client node in ```~/.local/share/juju/ssh/```
After deployment, Juju Controller is a "silent unit" that acts as a proxy between the client and the deployed applications. Nevertheless it can be useful to SSH into it.
First you need to understand your environment, especially if you run several Juju models and controllers. Run
```
juju list-models --all
$ juju models --all
Controller: k8s
Model Cloud/Region Status Machines Cores Access Last connection
admin/controller lxd/localhost available 1 - admin just now
admin/default lxd/localhost available 0 - admin 2017-01-23
admin/whale* lxd/localhost available 6 - admin 3 minutes ago
```
The first line ```Controller: k8s``` refers to how you bootstrapped.
Then you will see 2, 3 or more models listed below.
* admin/controller is the default model that hosts all controller units of juju
* admin/default is created by default as the primary model to host the user application, such as the Kubernetes cluster
* admin/whale is an additional model created if you use conjure-up as an overlay on top of Juju.
Now to ssh into a controller node, you first ask Juju to switch context, then ssh as you would with a normal unit:
```
juju switch controller
```
At this stage, you can query the controller model as well:
```
juju status
Model Controller Cloud/Region Version
controller k8s lxd/localhost 2.0.2
App Version Status Scale Charm Store Rev OS Notes
Unit Workload Agent Machine Public address Ports Message
Machine State DNS Inst id Series AZ
0 started 10.191.22.15 juju-2a5ed8-0 xenial
```
Note that if you had bootstrapped in HA mode, you would see several machines listed.
Now ssh-ing into the controller follows the same semantic as classic Juju commands:
```
$ juju ssh 0
Welcome to Ubuntu 16.04.1 LTS (GNU/Linux 4.8.0-34-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
0 packages can be updated.
0 updates are security updates.
Last login: Tue Jan 24 16:38:13 2017 from 10.191.22.1
ubuntu@juju-2a5ed8-0:~$
```
When you are done and want to come back to your initial model, exit the controller and
Then if you need to switch back to your cluster and ssh into the units, run
```
juju switch default
```
## Managing your Kubernetes cluster
### Running privileged containers
By default juju-deployed clusters do not support running privileged containers. If you need them, you have to edit ```/etc/default/kube-apiserver``` on the master nodes, and ```/etc/default/kubelet``` on your worker nodes.
On Kubernetes Core or on small deployment, run the following commands from the Juju client:
#### Manually
1. Update the Master
```
juju ssh kubernetes-master/0 "sudo sed -i 's/KUBE_API_ARGS=\"/KUBE_API_ARGS=\"--allow-privileged\ /' /etc/default/kube-apiserver && sudo systemctl restart kube-apiserver.service"
```
2. Update the Worker(s)
```
juju ssh kubernetes-worker/0 "sudo sed -i 's/KUBELET_ARGS=\"/KUBELET_ARGS=\"--allow-privileged\ /' /etc/default/kubelet && sudo systemctl restart kubelet.service"
```
#### Programmatically
If the deployment is larger the following commands will run on all units successively:
1. Update all Masters
```
juju show-status kubernetes-master --format json | \
jq --raw-output '.applications."kubernetes-master".units | keys[]' | \
xargs -I UNIT juju ssh UNIT "sudo sed -i 's/KUBE_API_ARGS=\"/KUBE_API_ARGS=\"--allow-privileged\ /' /etc/default/kube-apiserver && sudo systemctl restart kube-apiserver.service"
```
2. Update all workers
```
juju show-status kubernetes-worker --format json | \
jq --raw-output '.applications."kubernetes-worker".units | keys[]' | \
xargs -I UNIT juju ssh UNIT "sudo sed -i 's/KUBELET_ARGS=\"/KUBELET_ARGS=\"--allow-privileged\ /' /etc/default/kubelet && sudo systemctl restart kubelet.service"
```
{% endcapture %}
{% include templates/task.md %}
@@ -105,6 +105,102 @@ charm unit data, etc. Additional application-specific information may be
included as well.
## Common Problems
### Load Balancer interfering with Helm
This section assumes you have a working deployment of Kubernetes via Juju using a Load Balancer for the API, and that you are using Helm to deploy charts.
To deploy Helm you will have run:
```
helm init
$HELM_HOME has been configured at /home/ubuntu/.helm
Tiller (the helm server side component) has been installed into your Kubernetes Cluster.
Happy Helming!
```
Then when using helm you may see one of the following errors:
* Helm doesn't get the version from the Tiller server
```
helm version
Client: &version.Version{SemVer:"v2.1.3", GitCommit:"5cbc48fb305ca4bf68c26eb8d2a7eb363227e973", GitTreeState:"clean"}
Error: cannot connect to Tiller
```
* Helm cannot install your chart
```
helm install <chart> --debug
Error: forwarding ports: error upgrading connection: Upgrade request required
```
This is caused by the API load balancer not forwarding ports in the context of the helm client-server relationship. To deploy using helm, you will need to follow these steps:
1. Expose the Kubernetes Master service
```
juju expose kubernetes-master
```
2. Identify the public IP address of one of your masters
```
juju status kubernetes-master
Model Controller Cloud/Region Version
production k8s-admin aws/us-east-1 2.0.0
App Version Status Scale Charm Store Rev OS Notes
flannel 0.6.1 active 1 flannel jujucharms 7 ubuntu
kubernetes-master 1.5.1 active 1 kubernetes-master jujucharms 10 ubuntu exposed
Unit Workload Agent Machine Public address Ports Message
kubernetes-master/0* active idle 5 54.210.100.102 6443/tcp Kubernetes master running.
flannel/0 active idle 54.210.100.102 Flannel subnet 10.1.50.1/24
Machine State DNS Inst id Series AZ
5 started 54.210.100.102 i-002b7150639eb183b xenial us-east-1a
Relation Provides Consumes Type
certificates easyrsa kubernetes-master regular
etcd etcd flannel regular
etcd etcd kubernetes-master regular
cni flannel kubernetes-master regular
loadbalancer kubeapi-load-balancer kubernetes-master regular
cni kubernetes-master flannel subordinate
cluster-dns kubernetes-master kubernetes-worker regular
cni kubernetes-worker flannel subordinate
```
In this context the public IP address is 54.210.100.102.
If you want to access this data programmatically you can use the JSON output:
```
juju show-status kubernetes-master --format json | jq --raw-output '.applications."kubernetes-master".units | keys[]'
54.210.100.102
```
3. Update the kubeconfig file
Identify the kubeconfig file or section used for this cluster, and edit the server configuration.
By default, it will look like ```https://54.213.123.123:443```. Replace it with the Kubernetes Master endpoint ```https://54.210.100.102:6443``` and save.
Note that the default port used by CDK for the Kubernetes Master API is 6443 while the port exposed by the load balancer is 443.
4. Start helming again!
```
helm install <chart> --debug
Created tunnel using local port: '36749'
SERVER: "localhost:36749"
CHART PATH: /home/ubuntu/.helm/<chart>
NAME: <chart>
...
...
```
## etcd
@@ -88,7 +88,7 @@ If a proxy server is required, use the following method to start Minikube cluste
minikube start --vm-driver=xhyve --docker-env HTTP_PROXY=http://your-http-proxy-host:your-http-proxy-port --docker-env HTTPS_PROXY=http(s)://your-https-proxy-host:your-https-proxy-port
```
The `--vm-driver=xyhve` flag specifies that you are using Docker for Mac. The
The `--vm-driver=xhyve` flag specifies that you are using Docker for Mac. The
default VM driver is VirtualBox.
Now set the Minikube context. The context is what determines which cluster
+1 -1
View File
@@ -543,7 +543,7 @@ and need persistent storage, we recommend that you use the following pattern:
- Do include PersistentVolumeClaim objects in your bundle of config (alongside Deployments, ConfigMaps, etc).
- Do not include PersistentVolume objects in the config, since the user instantiating the config may not have
permission to create PersistentVolumes.
- Give the user the option of providing a storage class name when instantating the template.
- Give the user the option of providing a storage class name when instantiating the template.
- If the user provides a storage class name, and the cluster is version 1.4 or newer, put that value into the `volume.beta.kubernetes.io/storage-class` annotation of the PVC.
This will cause the PVC to match the right storage class if the cluster has StorageClasses enabled by the admin.
- If the user does not provide a storage class name or the cluster is version 1.3, then instead put a `volume.alpha.kubernetes.io/storage-class: default` annotation on the PVC.
+1 -1
View File
@@ -763,7 +763,7 @@ make that key begin with a dot. For example, when the following secret is mount
{
"name": "dotfile-test-container",
"image": "gcr.io/google_containers/busybox",
"command": "ls -l /etc/secret-volume",
"command": [ "ls", "-l", "/etc/secret-volume" ],
"volumeMounts": [
{
"name": "secret-volume",