Kubernetes GCP HTTP Load-Balancer Tips
We are in the midst of deploying a Kubernetes setup for one of our products on GCP using GKE and so far it’s been a great experience. I thought I would share a couple of tips around how the ingress resource works with the GCP HTTP(s) Load-balancer.
Google provides some great instructions on getting ingress up and running however we had a couple of things which I thought would be worth sharing to help others.
- Custom health check URL for GCP healtcheck resource
- Ingress with multiple clusters in a GCP project
- Firewall rules to your services
- Beware the Backend Services quota
Health check & cluster-uid
Fortunately the first two of these are features which can be enabled by manually editing the GLBC replication controller, you can the
cluster-uid example here.
You can then edit the args array in the yml to include the following:
Once saved delete the old pod and let the RC recreate it
uid can be anything you like in our case this references the environment. Once this is set all the various resources in your project will have
--uid apended to them allowing meaning cluster wont clash.
One gotcha about changing the healtcheck here is that this will apply to all you ingress resources, so you’ll need to make sure all your services return a 200 on this path.
The Google guide suggests configuring your firewall by extracting the NodePort assigned by Kubernetes and then creating firewall rules for this, but once you have more than a few services we have found it far easier to statically assign within a range and create a generic firewall rule which covers these and all future services.
For example if we assign our services to use
30000-31000 as their
We can then create a single run like this:
Backend Service Quota
Finally not really a tip as much as a heads up. Google by default gives you 3 Backend Services, however this is really inadequate once you start creating multiple services in your ingress resources.
You can request more, however it appears there is a cap of 30 per project without special approval.
I hope some of the manual step and limitations above will soon be replaced with automated options as Ingress moves from beta to production ready in
1.3, but even now in beta we have been impressed with how well everything fit together and works.