How To Launch Kubernetes Federation on Google Cloud Platform

How To Launch Kubernetes Federation on Google Cloud Platform

In this edition of our Kubernetes consulting series, we’ll take you through a step-by-step to launching Kubernetes Federation on Google Cloud Platform.

 

What is Kubernetes Federation? It allows you to combine several Kubernetes clusters and manage them through one Control Plane. With Federation, we’re able to synchronize resources in all clusters, reduce response time for queries from different parts of the world, and achieve high availability, as we place the clusters on different continents.

 

In our example, we will be using Google Kubernetes Engine.

DNS

Creating zone:

$ gcloud dns managed-zones create federation 
          --description "Kubernetes Federation Zone" 
            --dns-name federation.com

Checking:

$ gcloud dns managed-zones describe federation

 

Output:

creationTime: '2018-08-28T10:33:49.424Z'
description: Kubernetes Federation Zone
dnsName: federation.com.
id: '8875495119636580191'
kind: dns#managedZone
name: federation
nameServers:
- ns-cloud-e1.googledomains.com.
- ns-cloud-e2.googledomains.com.
- ns-cloud-e3.googledomains.com.
- ns-cloud-e4.googledomains.com.

Clusters

Creating a cluster in Asia:

$ gcloud container clusters create asia 
            --zone asia-southeast1-a 
            --scopes "cloud-platform,storage-ro,logging-write,monitoring-write,service-control,service-management,https://www.googleapis.com/auth/ndev.clouddns.readwrite"

Output:

Creating cluster asia...⠹
kubeconfig entry generated for asia.
NAME  LOCATION           MASTER_VERSION  MASTER_IP       MACHINE_TYPE   NODE_VERSION  NUM_NODES  STATUS
asia  asia-southeast1-a  1.9.7-gke.6     35.197.139.197  n1-standard-1  1.9.7-gke.6   3          RUNNING

Getting the connection credentials:

$ gcloud container clusters get-credentials asia 
            --zone asia-southeast1-a

Output:

Fetching cluster endpoint and auth data.
kubeconfig entry generated for asia.

Defining user policy:

$ kubectl create clusterrolebinding cluster-admin-binding 
            --clusterrole cluster-admin --user $(gcloud config get-value account)

Output:

Fetching cluster endpoint and auth data.
kubeconfig entry generated for asia.

Creating a cluster in Europe:

$ gcloud container clusters create europe 
            --zone europe-west2-a 
            --scopes "cloud-platform,storage-ro,logging-write,monitoring-write,service-control,service-management,https://www.googleapis.com/auth/ndev.clouddns.readwrite"

Getting the connection credentials:

$ gcloud container clusters get-credentials europe 
            --zone europe-west2-a

Defining user policy:

$ kubectl create clusterrolebinding cluster-admin-binding 
            --clusterrole cluster-admin --user $(gcloud config get-value account)

Creating a cluster in USA:

$ gcloud container clusters create america 
            --zone us-central1-a 
            --scopes "cloud-platform,storage-ro,logging-write,monitoring-write,service-control,service-management,https://www.googleapis.com/auth/ndev.clouddns.readwrite"

Getting the connection credentials:

$ gcloud container clusters get-credentials america 
            --zone us-central1-a

Defining user policy:

$ kubectl create clusterrolebinding cluster-admin-binding 
            --clusterrole cluster-admin --user $(gcloud config get-value account)

Let’s do the same thing again for two more clusters in Europe and Asia:

$ gcloud container clusters create asia-2 
            --zone asia-east1-a 
            --scopes "cloud-platform,storage-ro,logging-write,monitoring-write,service-control,service-management,https://www.googleapis.com/auth/ndev.clouddns.readwrite"
 
 
$ gcloud container clusters get-credentials asia-2 
            --zone asia-east1-a
 
$ kubectl create clusterrolebinding cluster-admin-binding 
            --clusterrole cluster-admin --user $(gcloud config get-value account)
$ gcloud container clusters create europe-2 
            --zone europe-north1-a 
            --scopes "cloud-platform,storage-ro,logging-write,monitoring-write,service-control,service-management,https://www.googleapis.com/auth/ndev.clouddns.readwrite"
 
$ gcloud container clusters get-credentials europe-2 
            --zone europe-north1-a 
 
$ kubectl create clusterrolebinding cluster-admin-binding 
            --clusterrole cluster-admin --user $(gcloud config get-value account)

Checking after all the actions are completed:

$ gcloud container clusters list

Output:

NAME      LOCATION           MASTER_VERSION  MASTER_IP       MACHINE_TYPE   NODE_VERSION  NUM_NODES  STATUS
asia-2    asia-east1-a       1.9.7-gke.6     35.221.221.46   n1-standard-1  1.9.7-gke.6   3          RUNNING
asia      asia-southeast1-a  1.9.7-gke.6     35.197.139.197  n1-standard-1  1.9.7-gke.6   3          RUNNING
europe-2  europe-north1-a    1.9.7-gke.6     35.228.203.204  n1-standard-1  1.9.7-gke.6   3          RUNNING
europe    europe-west2-a     1.9.7-gke.6     35.242.178.241  n1-standard-1  1.9.7-gke.6   3          RUNNING
america   us-central1-a      1.9.7-gke.6     35.188.203.7    n1-standard-1  1.9.7-gke.6   3          RUNNING

Or in GCP console:

Federation

Federation Control Plane manages the state of all your clusters. The control panel can be placed inside one of your Kubernetes clusters.

Even if the Control Plane cluster does not work, the other clusters are independent, so they will continue to function until the control panel goes back online. You can manage clusters separately! This means that you do not need to worry about one point of failure.

Now let’s see what contexts are used:

$ kubectl config get-contexts

Output:

          gke_federation_asia-east1-a_asia-2        gke_federation_asia-east1-a_asia-2        gke_federation_asia-east1-a_asia-2
*         gke_federation_europe-north1-a_europe-2   gke_federation_europe-north1-a_europe-2   gke_federation_europe-north1-a_europe-2

Kubernetes Federation uses the context name to create the Federation, but it must conform to the RFC1123 specification. It means that you need to rename the context. You can do this with the following commands:

$ kubectl config set-context asia 
   --cluster gke_federation_asia-southeast1-a_asia 
   --user gke_federation_asia-southeast1-a_asia
 
$ kubectl config delete-context 
            gke_federation_asia-southeast1-a_asia
 
$ kubectl config set-context europe 
   --cluster gke_federation_europe-west2-a_europe 
   --user gke_federation_europe-west2-a_europe
 
$ kubectl config delete-context 
            gke_federation_europe-west2-a_europe
 
$ kubectl config set-context america 
   --cluster gke_federation_us-central1-a_america 
   --user gke_federation_us-central1-a_america 
 
$ kubectl config delete-context 
            gke_federation_us-central1-a_america
 
$ kubectl config set-context asia-2 
   --cluster gke_federation_asia-east1-a_asia-2 
   --user gke_federation_asia-east1-a_asia-2
 
$ kubectl config delete-context 
            gke_federation_asia-east1-a_asia-2
 
$ kubectl config set-context europe-2 
   --cluster gke_federation_europe-north1-a_europe-2 
   --user gke_federation_europe-north1-a_europe-2
 
$ kubectl config delete-context 
            gke_federation_europe-north1-a_europe-2

Checking context:

$ kubectl config get-contexts

Output:

<span class="pln">CURRENT   NAME       CLUSTER                                           AUTHINFO                                          NAMESPACE
          america    gke_federation_us</span><span class="pun">-</span><span class="pln">central1</span><span class="pun">-</span><span class="pln">a_america      gke_federation_us</span><span class="pun">-</span><span class="pln">central1</span><span class="pun">-</span><span class="pln">a_america
          asia       gke_federation_asia</span><span class="pun">-</span><span class="pln">southeast1</span><span class="pun">-</span><span class="pln">a_asia     gke_federation_asia</span><span class="pun">-</span><span class="pln">southeast1</span><span class="pun">-</span><span class="pln">a_asia
          asia</span><span class="pun">-</span><span class="lit">2</span><span class="pln">     gke_federation_asia</span><span class="pun">-</span><span class="pln">east1</span><span class="pun">-</span><span class="pln">a_asia</span><span class="pun">-</span><span class="lit">2</span><span class="pln">        gke_federation_asia</span><span class="pun">-</span><span class="pln">east1</span><span class="pun">-</span><span class="pln">a_asia</span><span class="pun">-</span><span class="lit">2</span><span class="pln">
          europe     gke_federation_europe</span><span class="pun">-</span><span class="pln">west2</span><span class="pun">-</span><span class="pln">a_europe      gke_federation_europe</span><span class="pun">-</span><span class="pln">west2</span><span class="pun">-</span><span class="pln">a_europe
          europe</span><span class="pun">-</span><span class="lit">2</span><span class="pln">   gke_federation_europe</span><span class="pun">-</span><span class="pln">north1</span><span class="pun">-</span><span class="pln">a_europe</span><span class="pun">-</span><span class="lit">2</span><span class="pln">   gke_federation_europe</span><span class="pun">-</span><span class="pln">north1</span><span class="pun">-</span><span class="pln">a_europe</span><span class="pun">-</span><span class="lit">2</span>

To create a Federation we will use Kubefed. But it only works on Linux, it won’t start on Mac right away, but we can fix this with the Docker.

 

For authorization in Kubernetes on Google Cloud, we use google sdk. -/.kube /config:

- name: gke_krusche-federation_asia-southeast1-a_asia
  user:
    auth-provider:
      config:
        access-token: ya29.GlwHBt-fKASzD91Uxp-mtfbMMHz94w
        cmd-args: config config-helper --format=json
        cmd-path: /Users/roman/work/google-cloud-sdk/bin/gcloud
        expiry: 2018-08-28 15:03:09
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp

In my case, the binary file is located in my home directory, and Kubefed uses the same configuration files. Therefore, we will mount the entire home directory in the Docker container:

 

Dockerfile:

FROM centos:7
 
COPY bin/kubefed /usr/local/bin
COPY repo/kubernetes.repo /etc/yum.repos.d/
RUN mkdir -p /Users/roman 
    && yum install -y kubectl
 
ENV HOME /Users/roman
WORKDIR /Users/roman
 
ENTRYPOINT ["kubefed"]

Building Docker image:

$ docker build --no-cache --rm -t k8s/kubefed .

Defining alias:

alias kubefed='docker run -v "$HOME":/Users/roman k8s/kubefed'

Initializing Federation:

$ kubefed init kfed 
  --host-cluster-context=america 
  --dns-zone-name="federation.com." 
  --dns-provider="google-clouddns"

Output:

Creating a namespace federation-system for federation system components... done
Creating federation control plane service............. done
Creating federation control plane objects (credentials, persistent volume claim)... done
Creating federation component deployments... done
Updating kubeconfig... done
Waiting for federation control plane to come up................. done
Federation API server is running at: 104.154.131.222

Connecting clusters to the Federation Control Plane:

kubefed --context=kfed join asia 
  --cluster-context=asia 
  --host-cluster-context=america
 
kubefed --context=kfed join europe 
  --cluster-context=europe 
  --host-cluster-context=america
 
kubefed --context=kfed join america 
  --cluster-context=america 
  --host-cluster-context=america
 
kubefed --context=kfed join asia-2 
  --cluster-context=asia-2 
  --host-cluster-context=america
 
kubefed --context=kfed join europe-2 
  --cluster-context=europe-2 
  --host-cluster-context=america

Output:

cluster "asia" created
cluster "europe" created
cluster "america" created
cluster "asia-2" created
cluster "europe-2" created

Checking:

$ kubectl --context kfed get cluster

Output:

NAME       AGE
america    58s
asia       2m
asia-2     46s
europe     1m
europe-2   36s

Creating default namespace:

$ kubectl --context=kfed create ns default

Setting up and running the application

Creating a global static IP address:

$ gcloud compute addresses create ingress --global

Launching NGINX:

$ kubectl --context=kfed create deployment nginx --image=nginx:stable 
  && kubectl --context=kfed scale deployment nginx --replicas=12

Checking:

Creating NGINX Service:

$ kubectl --context=kfed create service nodeport nginx 
  --tcp=80:80 --node-port=30036

Creating file ingress.yaml:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx
  annotations:
    kubernetes.io/ingress.global-static-ip-name: ingress
spec:
  backend:
    serviceName: nginx
    servicePort: 80

Let’s deploy ingress:

kubectl --context=kfed create -f ingress.yaml

Ingress may not be created due to bugs 242 and 245.

 

In that case, we’ll create the balancer manually:

If everything’s ok, our NGINX should open.

 

Now let’s write a simple website with a world map to check the response of the clusters. You can check external IP from containers, but all of the Google IPs are American, so we’ll take the region from the cluster’s name.

 

Creating location.php:

<?php
$a = $_ENV[MY_NODE_NAME];
if (strpos($a, 'asia') !== false) {
  $continent = 'asia';
  $image = 'asia.png';
} elseif (strpos($a, 'europe') !== false) {
  $continent = 'europe';
  $image = 'europe.png';
} elseif (strpos($a, 'america') !== false) {
  $continent = 'North America';
  $image = 'america.png';
}
?>

Dockerfile:

FROM php:7.0-apache-stretch
ADD data /data
RUN cp -R /data/* /var/www/html 
    && chown www-data -R /var/www/html/

Building image:

docker build --no-cache --rm -t wacken/location:nodename .

Creating deployment.yaml:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 20
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - env:
        - name: MY_NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        name: nginx
        image: "wacken/location:nodename"
        imagePullPolicy: IfNotPresent
        ports:
          - containerPort: 80

Applying:

$ kubectl --context=kfed apply -f deployment.yaml

Checking

Now let’s open our balancer:

That’s it! Hurray!

 

So now we know how to setup Kubernetes Federation to reduce response time and ensure high availability of services. For more on Kubernetes setup you can also check out our step-by-step guide to How to Setup Kubernetes cluster on AWS.

 

If your organisation has a need Kubernetes consultancy or development resources please don’t hesitate to get in touch. K&C is one of Munich and Germany’s most trusted IT services providers with over 20 years of experience. We work with some of Europe’s best known brands, exciting start-ups and ambitious SMEs.

Add comment

E-mail is already registered on the site. Please use the Login form or enter another.

You entered an incorrect username or password

Sorry that something went wrong, repeat again!
Contact us