whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #20501 from onlydole/hotfix/gloo-image-updates

Browse Source 
Gloo post hotfixes
This commit is contained in:
Kubernetes Prow Robot
2020-04-22 15:41:45 -07:00
committed by GitHub
parent d6df57a88d 93ed9ab85f
commit 16e460faa1
1 changed files with 41 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
content/en/blog/_posts/2020-04-22-two-phased-canary-rollout-with-gloo.md
+41 -36
View File
@@ -17,8 +17,9 @@ How do we bring a new application online? How do we upgrade an application? How
platform, ops, and development teams?
In this post, we're going to use Gloo to design a two-phased canary rollout workflow for application upgrades:
* In the first phase, we'll do canary testing by shifting a small subset of traffic to the new version. This allows you to safely perform smoke and correctness tests.
* In the second phase, we'll progressively shift traffic to the new version, allowing us to monitor the new version under load, and eventually, decommission the old version.
- In the first phase, we'll do canary testing by shifting a small subset of traffic to the new version. This allows you to safely perform smoke and correctness tests.
- In the second phase, we'll progressively shift traffic to the new version, allowing us to monitor the new version under load, and eventually, decommission the old version.
To keep it simple, we're going to focus on designing the workflow using [open source Gloo](https://github.com/solo-io/gloo), and we're going to deploy the gateway and
application to Kubernetes. At the end, we'll talk about a few extensions and advanced topics that could be interesting to explore in a follow up.
@@ -30,10 +31,10 @@ features, and can be run against a local test cluster such as [minikube](https:/
This post assumes a basic understanding of Kubernetes and how to interact with it using `kubectl`.
We'll install the latest [open source Gloo](https://github.com/solo-io/gloo) to the `gloo-system` namespace and deploy
version `v1` of an example application to the `echo` namespace. We'll expose this application outside the cluster
by creating a route in Gloo, to end up with a picture like this:
version `v1` of an example application to the `echo` namespace. We'll expose this application outside the cluster
by creating a route in Gloo, to end up with a picture like this:
![Setup](/static/images/blog/2020-04-17-two-phased-canary-rollout-with-gloo/setup.png)
![Setup](/images/blog/2020-04-22-two-phased-canary-rollout-with-gloo/setup.png)
### Deploying Gloo
@@ -88,11 +89,12 @@ shifting traffic to a `v2` version of the application.
Kubernetes gives us a lot of flexibility in terms of modeling this application. We'll adopt the following
conventions:
* We'll include the version in the deployment name so we can run two versions of the application
side-by-side and manage their lifecycle differently.
* We'll label pods with an app label (`app: echo`) and a version label (`version: v1`) to help with our canary rollout.
* We'll deploy a single Kubernetes `Service` for the application to set up networking. Instead of updating
this or using multiple services to manage routing to different versions, we'll manage the rollout with Gloo configuration.
- We'll include the version in the deployment name so we can run two versions of the application
side-by-side and manage their lifecycle differently.
- We'll label pods with an app label (`app: echo`) and a version label (`version: v1`) to help with our canary rollout.
- We'll deploy a single Kubernetes `Service` for the application to set up networking. Instead of updating
this or using multiple services to manage routing to different versions, we'll manage the rollout with Gloo configuration.
The following is our `v1` echo application:
@@ -148,6 +150,7 @@ kubectl apply -f https://raw.githubusercontent.com/solo-io/gloo-ref-arch/blog-30
```
We should see the following output:
```
namespace/echo created
deployment.apps/echo-v1 created
@@ -155,6 +158,7 @@ service/echo created
```
And we should be able to see all the resources healthy in the `echo` namespace:
```
➜ kubectl get all -n echo
NAME READY STATUS RESTARTS AGE
@@ -210,7 +214,7 @@ metadata:
spec:
virtualHost:
domains:
- '*'
- "*"
routes:
- matchers:
- prefix: /
@@ -237,14 +241,14 @@ version:v1
Our setup is complete, and our cluster now looks like this:
![Setup](/static/images/blog/2020-04-17-two-phased-canary-rollout-with-gloo/setup.png)
![Setup](/images/blog/2020-04-22-two-phased-canary-rollout-with-gloo/setup.png)
## Two-Phased Rollout Strategy
Now we have a new version `v2` of the echo application that we wish to roll out. We know that when the
rollout is complete, we are going to end up with this picture:
![End State](/static/images/blog/2020-04-17-two-phased-canary-rollout-with-gloo/end-state.png)
![End State](/images/blog/2020-04-22-two-phased-canary-rollout-with-gloo/end-state.png)
However, to get there, we may want to perform a few rounds of testing to ensure the new version of the application
meets certain correctness and/or performance acceptance criteria. In this post, we'll introduce a two-phased approach to
@@ -268,7 +272,7 @@ In this phase, we'll deploy `v2`, and then use a header `stage: canary` to start
traffic to the new version. We'll use this header to perform some basic smoke testing and make sure `v2` is working the
way we'd expect:
![Subset Routing](/static/images/blog/2020-04-17-two-phased-canary-rollout-with-gloo/subset-routing.png)
![Subset Routing](/images/blog/2020-04-22-two-phased-canary-rollout-with-gloo/subset-routing.png)
### Setting up subset routing
@@ -284,7 +288,7 @@ metadata:
spec:
virtualHost:
domains:
- '*'
- "*"
routes:
- matchers:
- prefix: /
@@ -382,7 +386,7 @@ metadata:
spec:
virtualHost:
domains:
- '*'
- "*"
routes:
- matchers:
- headers:
@@ -464,7 +468,7 @@ metadata:
spec:
virtualHost:
domains:
- '*'
- "*"
routes:
# We'll keep our route from before if we want to continue testing with this header
- matchers:
@@ -502,17 +506,17 @@ spec:
values:
version: v2
weight: 0
```
We can apply this virtual service update to the cluster with the following commands:
```
kubectl apply -f https://raw.githubusercontent.com/solo-io/gloo-ref-arch/blog-30-mar-20/platform/prog-delivery/two-phased-with-os-gloo/3-progressive-traffic-shift-to-v2/vs-1.yaml
```
Now the cluster looks like this, for any request that doesn't have the `stage: canary` header:
![Initialize Traffic Shift](/static/images/blog/2020-04-17-two-phased-canary-rollout-with-gloo/init-traffic-shift.png)
![Initialize Traffic Shift](/images/blog/2020-04-22-two-phased-canary-rollout-with-gloo/init-traffic-shift.png)
With the initial weights, we should see the gateway continue to serve `v1` for all traffic.
@@ -525,7 +529,7 @@ version:v1
To simulate a load test, let's shift half the traffic to `v2`:
![Load Test](/static/images/blog/2020-04-17-two-phased-canary-rollout-with-gloo/load-test.png)
![Load Test](/images/blog/2020-04-22-two-phased-canary-rollout-with-gloo/load-test.png)
This can be expressed on our virtual service by adjusting the weights:
@@ -538,7 +542,7 @@ metadata:
spec:
virtualHost:
domains:
- '*'
- "*"
routes:
- matchers:
- headers:
@@ -607,7 +611,7 @@ We will save these topics for a future post on advanced canary testing use cases
We will continue adjusting weights until eventually, all of the traffic is now being routed to `v2`:
![Final Shift](/static/images/blog/2020-04-17-two-phased-canary-rollout-with-gloo/final-shift.png)
![Final Shift](/images/blog/2020-04-22-two-phased-canary-rollout-with-gloo/final-shift.png)
Our virtual service will look like this:
@@ -620,7 +624,7 @@ metadata:
spec:
virtualHost:
domains:
- '*'
- "*"
routes:
- matchers:
- headers:
@@ -661,6 +665,7 @@ spec:
```
We can apply that to the cluster with the following command:
```
kubectl apply -f https://raw.githubusercontent.com/solo-io/gloo-ref-arch/blog-30-mar-20/platform/prog-delivery/two-phased-with-os-gloo/3-progressive-traffic-shift-to-v2/vs-3.yaml
```
@@ -693,7 +698,7 @@ metadata:
spec:
virtualHost:
domains:
- '*'
- "*"
routes:
- matchers:
- prefix: /
@@ -721,7 +726,7 @@ kubectl delete deploy -n echo echo-v1
Now our cluster looks like this:
![End State](/static/images/blog/2020-04-17-two-phased-canary-rollout-with-gloo/end-state.png)
![End State](/images/blog/2020-04-22-two-phased-canary-rollout-with-gloo/end-state.png)
And requests to the gateway return this:
@@ -736,26 +741,26 @@ We have now completed our two-phased canary rollout of an application update usi
Over the course of this post, we collected a few topics that could be a good starting point for advanced exploration:
* Using the **JWT** filter to verify JWTs, extract claims onto headers, and route to canary versions depending on a claim value.
* Looking at **Prometheus metrics** and **Grafana dashboards** created by Gloo to monitor the health of the rollout.
* Automating the rollout by integrating **Flagger** with **Gloo**.
- Using the **JWT** filter to verify JWTs, extract claims onto headers, and route to canary versions depending on a claim value.
- Looking at **Prometheus metrics** and **Grafana dashboards** created by Gloo to monitor the health of the rollout.
- Automating the rollout by integrating **Flagger** with **Gloo**.
A few other topics that warrant further exploration:
* Supporting **self-service** upgrades by giving teams ownership over their upstream and route configuration
* Utilizing Gloo's **delegation** feature and Kubernetes **RBAC** to decentralize the configuration management safely
* Fully automating the continuous delivery process by applying **GitOps** principles and using tools like **Flux** to push config to the cluster
* Supporting **hybrid** or **non-Kubernetes** application use-cases by setting up Gloo with a different deployment pattern
* Utilizing **traffic shadowing** to begin testing the new version with realistic data before shifting production traffic to it
- Supporting **self-service** upgrades by giving teams ownership over their upstream and route configuration
- Utilizing Gloo's **delegation** feature and Kubernetes **RBAC** to decentralize the configuration management safely
- Fully automating the continuous delivery process by applying **GitOps** principles and using tools like **Flux** to push config to the cluster
- Supporting **hybrid** or **non-Kubernetes** application use-cases by setting up Gloo with a different deployment pattern
- Utilizing **traffic shadowing** to begin testing the new version with realistic data before shifting production traffic to it
## Get Involved in the Gloo Community
Gloo has a large and growing community of open source users, in addition to an enterprise customer base. To learn more about
Gloo:
* Check out the [repo](https://github.com/solo-io/gloo), where you can see the code and file issues
* Check out the [docs](https://docs.solo.io/gloo/latest), which have an extensive collection of guides and examples
* Join the [slack channel](http://slack.solo.io/) and start chatting with the Solo engineering team and user community
- Check out the [repo](https://github.com/solo-io/gloo), where you can see the code and file issues
- Check out the [docs](https://docs.solo.io/gloo/latest), which have an extensive collection of guides and examples
- Join the [slack channel](http://slack.solo.io/) and start chatting with the Solo engineering team and user community
If you'd like to get in touch with me (feedback is always appreciated!), you can find me on the
[Solo slack](http://slack.solo.io/) or email me at **rick.ducott@solo.io**.