From 0c5c99fd0dc6d2a1c12e594cbef39d4eda4cdb76 Mon Sep 17 00:00:00 2001 From: "Madhusudan.C.S" Date: Tue, 22 Nov 2016 12:55:25 -0800 Subject: [PATCH 1/3] Document the workaround for the Federated Ingress flapping healthchecks problem. Ref issue: https://github.com/kubernetes/kubernetes/issues/36327 and https://github.com/kubernetes/kubernetes/issues/37306 --- .../federation/federated-ingress.md | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/docs/user-guide/federation/federated-ingress.md b/docs/user-guide/federation/federated-ingress.md index 87965a3fc7..bd45316883 100644 --- a/docs/user-guide/federation/federated-ingress.md +++ b/docs/user-guide/federation/federated-ingress.md @@ -229,6 +229,38 @@ kept running, the Federated Ingress ensures that user traffic is automatically redirected away from the failed cluster to other available clusters. +## Known issue + +GCE L7 load balancer backends and healthchecks are known to flap due +to the firewall rules from the underlying clusters in federation +clobbering each other. In order to workaround this problem, you can +install the firewall rules manually to expose the targets of all the +underlying clusters in your federation for each Federated Ingress +object so that the health checks can pass and GCE L7 load balancer +is stable. This can be done using the `gcloud` command line tool as follows: + +```shell +gcloud compute firewall-rules create \ + --source-ranges 130.211.0.0/22 --allow [] \ + --target-tags [] \ + --network +``` + +where: +1. `firewall-rule-name` can be any name. +2. `[]` is the comma separated list of node ports corresponding to the services that backs the Federated Ingress. +3. [] is the comma separated list of the target tags assigned to the nodes in a kubernetes cluster. +4. is the name of the network where the firewall rule must be installed. + +Example: +```shell +gcloud compute firewall-rules create my-federated-ingress-firewall-rule \ + --source-ranges 130.211.0.0/22 --allow tcp:30301, tcp:30061, tcp:34564 \ + --target-tags my-cluster-1-minion, my-cluster-2-minion \ + --network default +``` + + ## Troubleshooting #### I cannot connect to my cluster federation API From 715d5a3c968a3c52d4afbba4ec0774cc260869fe Mon Sep 17 00:00:00 2001 From: "Madhusudan.C.S" Date: Wed, 23 Nov 2016 00:34:02 -0800 Subject: [PATCH 2/3] Address review comments. --- docs/user-guide/federation/federated-ingress.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/federation/federated-ingress.md b/docs/user-guide/federation/federated-ingress.md index bd45316883..2b501a5ebe 100644 --- a/docs/user-guide/federation/federated-ingress.md +++ b/docs/user-guide/federation/federated-ingress.md @@ -237,7 +237,13 @@ clobbering each other. In order to workaround this problem, you can install the firewall rules manually to expose the targets of all the underlying clusters in your federation for each Federated Ingress object so that the health checks can pass and GCE L7 load balancer -is stable. This can be done using the `gcloud` command line tool as follows: +can remain stable. These rules can be installed using the +[`gcloud`](https://cloud.google.com/sdk/gcloud/) command line tool, +[Google Cloud Console](https://console.cloud.google.com) or the +[Google Compute Engine APIs](https://cloud.google.com/compute/docs/reference/latest/). + +You can install these rules using +[`gcloud`](https://cloud.google.com/sdk/gcloud/) as follows: ```shell gcloud compute firewall-rules create \ @@ -247,8 +253,9 @@ gcloud compute firewall-rules create \ ``` where: + 1. `firewall-rule-name` can be any name. -2. `[]` is the comma separated list of node ports corresponding to the services that backs the Federated Ingress. +2. `[]` is the comma separated list of node ports corresponding to the services that back the Federated Ingress. 3. [] is the comma separated list of the target tags assigned to the nodes in a kubernetes cluster. 4. is the name of the network where the firewall rule must be installed. From ea411fb18d33bb57ef5f290201786c9e8302ade4 Mon Sep 17 00:00:00 2001 From: devin-donnelly Date: Mon, 12 Dec 2016 14:41:51 -0500 Subject: [PATCH 3/3] Update federated-ingress.md --- docs/user-guide/federation/federated-ingress.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/user-guide/federation/federated-ingress.md b/docs/user-guide/federation/federated-ingress.md index 2b501a5ebe..dcd5b6b4d4 100644 --- a/docs/user-guide/federation/federated-ingress.md +++ b/docs/user-guide/federation/federated-ingress.md @@ -231,13 +231,12 @@ available clusters. ## Known issue -GCE L7 load balancer backends and healthchecks are known to flap due -to the firewall rules from the underlying clusters in federation -clobbering each other. In order to workaround this problem, you can +GCE L7 load balancer back-ends and health checks are known to "flap"; this is due +to conflicting firewall rules in the federation's underlying clusters, which might override one another. To work around this problem, you can install the firewall rules manually to expose the targets of all the underlying clusters in your federation for each Federated Ingress -object so that the health checks can pass and GCE L7 load balancer -can remain stable. These rules can be installed using the +object. This way, the health checks can consistently pass and the GCE L7 load balancer +can remain stable. You install the rules using the [`gcloud`](https://cloud.google.com/sdk/gcloud/) command line tool, [Google Cloud Console](https://console.cloud.google.com) or the [Google Compute Engine APIs](https://cloud.google.com/compute/docs/reference/latest/).