Istio Explained – Service Mesh Routing

In the first of our series of Kubernest Consulting series of posts covering ISTIO, we’ll look at exactly what Istio is and how the Google open source project brings the complexities of managing the networks used to connect microservices under control.

Over the past couple of years, a DevOps consulting issue we have encountered with increasing frequency is in-house DevOps or development resources struggling to effectively manage increasingly complex combinations of microservices. This is particularly often the case for organisations that have more recently transitioned to building apps on a microservice rather than monolith architecture. One of the most important tools we use to help manage the connections between microservices is Istio. We also provide a step-by-step tutorial on how to set Istio up and use it for traffic routing, switching, enabling timeouts and retries, mirroring and circuit breaker implementation.

Microservices – Business and Technology Pros and Cons

Microservices allow for big, complex apps to be broken up into independent, containerized services. These containers then form Kubernetes clusters. The advantage of this is a reduction in costs as initial development is simplified and so faster. Updates to existing apps or scaling them is also simplified – all of which boosts an organisation’s bottom line.

However, the decision to go with a microservices architecture inevitably introduces some challenges alongside the advantages. Breaking apps down into independent services means an increase in the number of moving parts that need to be connected and secured. Managing this network of independent services such as traffic management, load balance, authentication, authorization etc. etc., can, especially in the case of complex apps, become extremely complex.

The term for the ‘space’ that exists between Kubernetes containers in a cluster is a Service Mesh. Istio, is an open source project initiated by Google and also involving IBM and ride-share app tech company Lyft. It’s a tool to manage the Service Mesh of a Kubernetes cluster – taming it before it becomes a complex zone of chaos that is a potential source of bugs.

Istio Service Mesh

The Service Mesh is the architecture layer responsible for reliable delivery of requests through a complex network of microservices.

When your application has evolved from a monolith to a microservice architecture, it becomes increasingly difficult to manage and monitor it every day. In this case, you need to move on to solutions that solve some of the problems associated with microservices:

• load balancing inside the microservice grid;
• service discovery
• Failure recovery
• metrics
• monitoring

They also solve more complex problems:

• A / B testing;
• canary rollouts (Canary rollouts);
• access control
• end-to-end authentication

When to use Istio

This is where Istio comes to the rescue.

Key features of Istio:

• traffic management: timeouts, retries, load balancing;
• security: authentication and authorization;
• observability: trace, monitoring;

Istio Architecture

Istio Service Mesh is logically divided into data plane and control plane.

• Data plane – consists of a set of smart proxies deployed as sidecars. These proxies provide and control all network communication between services together with the Mixer center of policies and telemetry;
• Control Plane – Configures proxies for routing traffic. The control plane also configures the Mixer to apply policies and telemetry.

Istio Architecture (source)

Istio components

• Envoy is a high-performance proxy developed in C ++ to transfer all incoming and outgoing traffic for all services;
• Mixer – provides access control and usage policies for the network of services and collects telemetry data from the Envoy proxy server and other services;
• Pilot – provides service discovery for Envoy sidecar, provides opportunities for intelligent traffic routing (for example, A / B tests, canary deployments) and fault tolerance (timeouts, retries, circuit breakers);
• Citadel – provides reliable service-to-service and end-user authentication;
• Galley – is a component of Istio configuration validation. He is responsible for isolating the remaining components of Istio from the user configuration of the underlying platform.

Routing and traffic configuration in Istio

The Istio traffic routing and configuration model uses the following API resources:

• Virtual services – sets up rules for routing Envoy traffic inside our service mesh;
• Destination rules – sets up policies for after applying routing rules to Virtual services;
• Gateways – to configure the Envoy load balancing method (HTTP, TCP or gRPC);
• Service entries – to configure external grid dependencies.

Step-by-step tutorial on how to install and configure Istio

We will run istio on the Google Kubernetes Engine (GKE). Create a cluster:

gcloud container clusters create <cluster-name> \
  --cluster-version latest \
  --num-nodes 4 \
  --zone <zone> \
  --project <project-id>

We obtain access keys (hereinafter credentials):

gcloud container clusters get-credentials <cluster-name> \
--zone <zone> \
--project <project-id>

We give administrator rights to our user:

kubectl create clusterrolebinding cluster-admin-binding \
    --clusterrole=cluster-admin \
    --user=$(gcloud config get-value core/account)

After preparing the cluster, we proceed to install the Istio components. Download the latest version, at the time of writing, Istio version 1.3.0:

curl -L https://git.io/getLatestIstio | ISTIO_VERSION=1.3.0 sh -

Go to the directory with Istio:

cd istio-1.3.0

Install Custom Resource Definitions (CRDs) for Istio with kubectl

Install Custom Resource Definitions (CRDs) for Istio with kubectl

After installing the CRD, install Isito control plane itself with mutual TLS authentication:

kubectl apply -f install/kubernetes/istio-demo-auth.yaml

Checking services:

NAME                     TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                                                                                                                                      AGE
grafana                  ClusterIP      10.43.242.92    <none>         3000/TCP                                                                                                                                     4m1s
istio-citadel            ClusterIP      10.43.252.216   <none>         8060/TCP,15014/TCP                                                                                                                           3m58s
istio-egressgateway      ClusterIP      10.43.254.22    <none>         80/TCP,443/TCP,15443/TCP                                                                                                                     4m2s
istio-galley             ClusterIP      10.43.244.7     <none>         443/TCP,15014/TCP,9901/TCP                                                                                                                   4m3s
istio-ingressgateway     LoadBalancer   10.43.253.1     34.69.43.198   15020:31384/TCP,80:31380/TCP,443:31390/TCP,31400:31400/TCP,15029:30472/TCP,15030:32532/TCP,15031:30101/TCP,15032:30948/TCP,15443:30384/TCP   4m1s
istio-pilot              ClusterIP      10.43.250.244   <none>         15010/TCP,15011/TCP,8080/TCP,15014/TCP                                                                                                       3m59s
istio-policy             ClusterIP      10.43.242.33    <none>         9091/TCP,15004/TCP,15014/TCP                                                                                                                 4m
istio-sidecar-injector   ClusterIP      10.43.244.233   <none>         443/TCP,15014/TCP                                                                                                                            3m58s
istio-telemetry          ClusterIP      10.43.253.8     <none>         9091/TCP,15004/TCP,15014/TCP,42422/TCP                                                                                                       3m59s
jaeger-agent             ClusterIP      None            <none>         5775/UDP,6831/UDP,6832/UDP                                                                                                                   3m43s
jaeger-collector         ClusterIP      10.43.250.60    <none>         14267/TCP,14268/TCP                                                                                                                          3m43s
jaeger-query             ClusterIP      10.43.242.192   <none>         16686/TCP                                                                                                                                    3m43s
kiali                    ClusterIP      10.43.242.83    <none>         20001/TCP                                                                                                                                    4m
prometheus               ClusterIP      10.43.241.166   <none>         9090/TCP                                                                                                                                     3m59s
tracing                  ClusterIP      10.43.245.22    <none>         80/TCP                                                                                                                                       3m42s
zipkin                   ClusterIP      10.43.248.101   <none>         9411/TCP                                                                                                                                     3m42s

and pods:

NAME                                      READY   STATUS      RESTARTS   AGE
grafana-6fc987bd95-t4pwj                  1/1     Running     0          4m54s
istio-citadel-679b7c9b5b-tktt7            1/1     Running     0          4m48s
istio-cleanup-secrets-1.3.0-q9xrb         0/1     Completed   0          5m16s
istio-egressgateway-5db67796d5-pmcr2      1/1     Running     0          4m58s
istio-galley-7ff97f98b5-jn796             1/1     Running     0          4m59s
istio-grafana-post-install-1.3.0-blqtb    0/1     Completed   0          5m19s
istio-ingressgateway-859bb7b4-ms2zr       1/1     Running     0          4m56s
istio-pilot-9b9f7f5c8-7h4j7               2/2     Running     0          4m49s
istio-policy-754cbf67fb-5vk9f             2/2     Running     2          4m52s
istio-security-post-install-1.3.0-7wffc   0/1     Completed   0          5m15s
istio-sidecar-injector-68f4668959-ql975   1/1     Running     0          4m47s
istio-telemetry-7cf8dcfd54-crd9w          2/2     Running     2          4m50s
istio-tracing-669fd4b9f8-c8ptq            1/1     Running     0          4m46s
kiali-94f8cbd99-h4b5z                     1/1     Running     0          4m53s
prometheus-776fdf7479-krzqm               1/1     Running     0          4m48s

Application launch

We will use the lite version of bookinfo, written by K&C for testing Istio. We won’t use UI yet (it doesn’t work well enough for presentation).

Application Architecture:

• ui
• Gateway — API
• Books
• Ratings

Differences in application versions:

books:

• v1 —  no description
• v2 — description

ratings:

• v1 — presentation «:-)»
• v2 — presentation «¯\\_(ツ)_/¯»

Create a namespace for the application:

kubectl create ns mesh
kubectl label namespace mesh istio-injection=enabled

Create the books-deployment.yaml file with the contents.

This is a set of standard api deployments and services for deploying a regular application in kubernetes. In this example, we use one version of Gateway and 2 versions of books and ratings. In service selector is registered only to the name of the application, and not to the version; we will configure routing to a specific version using Istio

# ===========
# = Gateway =
# ===========
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gw
  labels:
    app: gw
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gw
      version: v1
  template:
    metadata:
      labels:
        app: gw
        version: v1
    spec:
      containers:
      - name: gw
        image: kruschecompany/mesh:gateway
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: gw
spec:
  selector:
    app: gw
  ports:
    - name: http
      port: 8080
 
# =========
# = Books = 
# =========
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: books-v1
  labels:
    app: books
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: books
      version: v1
  template:
    metadata:
      labels:
        app: books
        version: v1
    spec:
      containers:
      - name: books
        image: kruschecompany/mesh:books_v1
        ports:
        - containerPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: books-v2
  labels:
    app: books
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: books
      version: v2
  template:
    metadata:
      labels:
        app: books
        version: v2
    spec:
      containers:
      - name: books
        image: kruschecompany/mesh:books_v2
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: books
spec:
  selector:
    app: books
  ports:
    - name: http
      port: 80
      targetPort: 8080
 
 
# ===========
# = Ratings =
# ===========
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ratings-v1
  labels:
    app: ratings
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ratings
      version: v1
  template:
    metadata:
      labels:
        app: ratings
        version: v1
    spec:
      containers:
      - name: ratings
        image: kruschecompany/mesh:ratings_v1
        ports:
        - containerPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ratings-v2
  labels:
    app: ratings
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ratings
      version: v2
  template:
    metadata:
      labels:
        app: ratings
        version: v2
    spec:
      containers:
      - name: ratings
        image: kruschecompany/mesh:ratings_v2
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: ratings
spec:
  selector:
    app: ratings
  ports:
    - name: http
      port: 80
      targetPort: 8080

We apply:

kubectl -n mesh apply -f books-deployment.yaml

We check:

kubectl -n mesh get pod

NAME                          READY   STATUS    RESTARTS   AGE
books-v1-758875cb99-sj4wm     2/2     Running   0          26m
books-v2-64c4889569-jjpnt     2/2     Running   0          26m
gw-7488b5dcbd-2t9xr           2/2     Running   0          26m
ratings-v1-57f7d99c55-kxnm7   2/2     Running   0          26m
ratings-v2-5d856c95d5-dm2tk   2/2     Running   0          26m

In the conclusion, we see 2 containers running in each pod, Istio made an injection container with Envoy during the deployment, in the future all traffic will go through these containers.

We create the istio-gateway.yaml file with the contents. Istio does not allow the use of wildcard in VirtualService, so replace ‘*’ with the balancer ip:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mesh
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: mesh-gw
spec:
  hosts:
  - "*" # <- replace on load balancer ip
  gateways:
  - mesh
  http:
  - match:
    - uri:
        exact: /gateway/books
    route:
    - destination:
        port:
          number: 8080
        host: gw

We apply:

kubectl -n mesh apply -f istio-gateway.yaml

We determined the entry point to our application, all incoming traffic from uri / gateway / books will be routed to the gateway service (aka gw).

Now create the istio-destinationrule.yaml file:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: gw
spec:
  host: gw
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
  subsets:
  - name: v1
    labels:
      version: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: books
spec:
  host: books
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: ratings
spec:
  host: ratings
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

In the subset, we determined where to route traffic, for gw traffic will go to under c version 1, books and ratings for both versions in turn.

We apply:

kubectl -n mesh apply -f istio-destinationrule.yaml

We open in the browser http: // <load_balancer_ip> / gateway / books is our API. We get JSON with books and their rating.

[
{
"id": 1,
"name": "War and Piece",
"rating": 455.45,
"presentation": ":-)",
"description": "Historical"
},
{
"id": 2,
"name": "Anna Karenina",
"rating": 666.4,
"presentation": ":-)",
"description": "Drama"
},
...
]

Try to refresh the page – the output will be different because the application will access different services each time.

[
{
"id": 1,
"name": "War and Piece",
"rating": 455.45,
"presentation": "¯\\_(ツ)_/¯",
"description": "Historical"
},
{
"id": 2,
"name": "Anna Karenina",
"rating": 666.4,
"presentation": "¯\\_(ツ)_/¯",
"description": "Drama"
},
...
]

And a couple of times more:

[
{
"id": 1,
"name": "War and Piece",
"rating": 455.45,
"presentation": ":-)",
"description": null
},
{
"id": 2,
"name": "Anna Karenina",
"rating": 666.4,
"presentation": ":-)",
"description": null
},
...
]

The application topology can also be viewed through Kiali, which was installed along with other components of Istio. To do this, use port-forward to forward the service to our machine:

kubectl -n istio-system port-forward $(kubectl -n istio-system get pod -l app=kiali -o jsonpath='{.items[0].metadata.name}') 20001:20001

Kiali will be available at http: // localhost: 20001 admin / admin. On the Graph tab we see:

Traffic routing

Everything is beautiful in the picture, but in real life you need traffic to go to certain versions of services. To do this, create another file istio-virtual-service-all-v1.yaml, in which we determine that all requests will go to books and rating version 1:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: books
spec:
  hosts:
  - books
  http:
  - route:
    - destination:
        host: books
        subset: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
  - ratings
  http:
  - route:
    - destination:
        host: ratings
        subset: v1

And apply:

kubectl -n mesh apply -f istio-virtual-service-all-v1.yaml

We check that the browser should see the same output:

[
{
"id": 1,
"name": "War and Piece",
"rating": 455.45,
"presentation": ":-)",
"description": null
},
{
"id": 2,
"name": "Anna Karenina",
"rating": 666.4,
"presentation": ":-)",
"description": null
},
...
]

In this example, in the subset we specified only v1 for books and ratings, and all traffic went only to the first version.

Traffic switching

We apply weight-based routing. This means that we put weights on services. In the example, we put the first version 50 on the service ratings and the same on the second version. Now all traffic will be balanced 50/50 between versions. We can also supply 10/90, in which case 10% of the traffic will go to the first version and 90% to the second.

Create a virtual-service-ratings-50.yaml file:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
    - ratings
  http:
  - route:
    - destination:
        host: ratings
        subset: v1
      weight: 50
    - destination:
        host: ratings
        subset: v2
      weight: 50

We apply:

kubectl -n mesh apply -f virtual-service-ratings-50.yaml

Check on browser:

[
{
"id": 1,
"name": "War and Piece",
"rating": 455.45,
"presentation": ":-)",
"description": null
},
{
"id": 2,
"name": "Anna Karenina",
"rating": 666.4,
"presentation": ":-)",
"description": null
},
...
]

We refresh the page a couple of times:

[
{
"id": 1,
"name": "War and Piece",
"rating": 455.45,
"presentation": "¯\\_(ツ)_/¯",
"description": null
},
{
"id": 2,
"name": "Anna Karenina",
"rating": 666.4,
"presentation": "¯\\_(ツ)_/¯",
"description": null
},
...
]

Clean and move on to the next example:

kubectl -n mesh apply -f istio-virtual-service-all-v1.yaml

Timeouts and Retries

Istio allows you to enable timeouts for services, so you can artificially simulate the long response of microservices.

We create the istio-delay.yaml file, in which we set a delay of 2 seconds for all requests:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
  - ratings
  http:
  - fault:
      delay:
        percent: 100
        fixedDelay: 2s
    route:
    - destination:
        host: ratings
        subset: v1

We apply:

kubectl -n mesh apply -f istio-delay.yaml

We check in the browser – the application works, but with a delay. Increase the timeout to 5 seconds.

We apply and verify:

kubectl -n mesh apply -f istio-delay.yaml

We get an error in response, now we know that the application will crash if one of the microservices responds for a long time:

You can also add retries and a timeout for retry:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
  - ratings
  http:
    route:
    - destination:
        host: ratings
        subset: v1
    retries:
      attempts: 3
      perTryTimeout: 2s

Clean and move on to the next example:

kubectl -n mesh apply -f istio-virtual-service-all-v1.yaml

Traffic mirroring

Sometimes you need to check the new version with more users, but you can’t roll it out to the prod. To do this, Istio has traffic mirroring functionality, we launch a new version of the service in parallel and direct traffic there, without affecting the working version of the service.

To do this, create the file istio-mirroring.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
  - ratings
  http:
  - route:
    - destination:
        host: ratings
        subset: v1
      weight: 100
    mirror:
      host: ratings
      subset: v2

We apply:

kubectl -n mesh apply -f istio-mirroring.yaml

We’re checking:

while true;do curl http://<load_balancer_ip>/gateway/books; sleep 2;done

We get the answer from ratings of the first version:

[
  {
    "id": 1,
    "name": "War and Piece",
    "rating": 455.45,
    "presentation": ":-)",
    "description": null
  },
  {
    "id": 2,
    "name": "Anna Karenina",
    "rating": 666.4,
    "presentation": ":-)",
    "description": null
  },
...
]

In the ratings container logs of the second version, we see that traffic is mirrored to it:

2019-09-18 11:19:04.574  INFO 1 --- [nio-8080-exec-8] c.m.r.controller.BooksRatingsController  : [1, 2, 3, 4, 5, 6]
2019-09-18 11:19:06.686  INFO 1 --- [nio-8080-exec-9] c.m.r.controller.BooksRatingsController  : [1, 2, 3, 4, 5, 6]
2019-09-18 11:19:08.801  INFO 1 --- [io-8080-exec-10] c.m.r.controller.BooksRatingsController  : [1, 2, 3, 4, 5, 6]
2019-09-18 11:19:10.918  INFO 1 --- [nio-8080-exec-1] c.m.r.controller.BooksRatingsController  : [1, 2, 3, 4, 5, 6]
2019-09-18 11:19:13.065  INFO 1 --- [nio-8080-exec-2] c.m.r.controller.BooksRatingsController  : [1, 2, 3, 4, 5, 6]

Clean and move on to the next example:

kubectl -n mesh apply -f istio-virtual-service-all-v1.yaml

Circuit Breaker

It is very important that our requests are guaranteed to reach the addressee. Istio implements the Circuit Breaking mechanism. Proxies inside the cluster interrogate services and, in the event of a breakdown or slow response, turn off the service instance (sub) from the network and direct the load to other service replicas.

For the books service, the following rules apply:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: books
spec:
  host: books
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 1
      http:
        http1MaxPendingRequests: 1
        maxRequestsPerConnection: 1
    outlierDetection:
      consecutiveErrors: 1
      interval: 1s
      baseEjectionTime: 3m
      maxEjectionPercent: 100
    tls:
      mode: ISTIO_MUTUAL
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

• maxConnections – The maximum number of connections to the service. Any redundant connection will be in the queue.
• http1MaxPendingRequests – the maximum number of pending service requests. Any excess pending requests will be rejected.
• maxRequestsPerConnection – the maximum number of requests in the cluster.
• BaseEjectionTime – maximum extraction time for the hearth. The under will be removed for 20 seconds.
• ConsecutiveErrors – the number of errors before the sub is removed from the pool. For example, if you have three consecutive errors while interacting with a service, Istio marks it as unhealthy.
• Interval – time interval for outlier analysis. For example, a service is checked every 10 seconds.
• MaxEjectionPercent – the maximum percentage of hearths that can be extracted from the load balancing pool. For example, setting this field to 100 implies that any unhealthy hearths that produce consistent errors can be retrieved and the request will be redirected to healthy hearths.

And there you have it – a step-by-step technical guide to using Istio for routing in a microservices-based app environment.

K&C – IT Outsourcing You Can Trust

If your organization needs any assistance with Istio integration and set-up or any other Kubernetes or wider web development gaps in your in-house resource, K&C would be delighted to hear from you.

Based in and managed from Munich, Germany, our nearshored tech talent hubs in Kyiv and Krakow offer you access to some of Europe’s best developers, software testers and IT consultants at rates that are competitive without compromising quality.

Our dedicated teams, team extensions and consultants can either work with you onsite wherever you are in Germany or Europe, remotely from our K&C offices or as a hybrid model. We service your needs, as they are and as they change. Please do Get in Touch.