Introduction to Istio Traffic Management

1. Istio Traffic Management

Istio is an open-source service mesh that provides a network layer abstraction for distributed microservices architectures. It makes communication between services more reliable and secure, and offers fine-grained traffic management, monitoring, and policy enforcement capabilities. Istio achieves this by inserting a transparent proxy (Envoy) between services to intercept all network communication, allowing developers to focus on business logic without dealing with network issues such as service discovery, load balancing, fault recovery, and security.

The relationship between Istio and k8s is very close. It can leverage the powerful features of k8s to manage and deploy the service mesh. When Istio is deployed in a k8s cluster, it automatically detects services within the cluster and provides them with network layer abstraction. This allows developers to use Istio to manage communication between services while utilizing k8s for service deployment and scaling. This combination makes it easier for developers to build, deploy, and manage complex microservices architectures.

Istio traffic routing rules allow you to easily control traffic between services and API calls. Istio simplifies the configuration of service-level attributes such as circuit breakers, timeouts, and retries, and makes it easy to set up important tasks like A/B testing, canary releases, and staged rollouts based on traffic percentages. It also provides out-of-the-box fault recovery features that enhance application robustness, making it better able to handle failures of dependent services or networks.

Traffic management is one of the core functionalities of Istio, making network layer operations simpler and more flexible. Below is a detailed introduction to the core components and concepts of Istio traffic management:

• • Envoy Proxy: Envoy is a high-performance C++ distributed proxy designed for individual services and applications, as well as for large microservices “service mesh” architectures. It is deployed as a sidecar with the service container, intercepting all incoming and outgoing traffic without requiring any changes to the service code. Envoy supports protocols such as HTTP/2 and gRPC, and provides features like circuit breakers, anomaly detection, and rate limiting to ensure network stability and reliability.

• • Pilot: Pilot provides service discovery functionality for Istio. It converts service configurations from the service directory into configurations for Envoy proxies and ensures that these configurations are dynamically distributed to all Envoy proxies at runtime. Pilot abstracts various service discovery systems, such as Kubernetes and Consul, allowing Istio to operate in different environments.

• • Mixer: Mixer is responsible for enforcing access control and usage policies in the service mesh. It allows services to validate request attributes before performing operations and collects telemetry data (such as logs, metrics, and quotas). Mixer communicates with various backend infrastructures through adapters, such as Prometheus, StatsD, and MySQL.

• • Citadel: Citadel is responsible for the security features of Istio, including certificate management, key management, and secure communication between services. It provides automated key and certificate rotation for workloads in the service mesh, ensuring the security of end-to-end communication.

• • Virtual Service: A Virtual Service is an Istio configuration resource that allows users to define routing rules to control traffic routing between services. Through Virtual Service, users can set up condition matches (such as based on paths or HTTP headers) and redirect traffic to different versions or services.

• • Destination Rule: A Destination Rule works in conjunction with Virtual Service to define traffic policies between service versions, such as load balancing policies, connection pool sizes, circuit breaker settings, etc.

• • Gateway: A Gateway is an Istio configuration resource used to manage ingress and egress traffic in the service mesh. It allows users to configure load balancers for the service mesh to handle traffic from external networks.

• • Service Entry: A Service Entry is used to add external services to Istio‘s service directory, allowing services within the mesh to access external services while still leveraging Istio‘s traffic management capabilities.

Through these core components and concepts, Istio provides a unified way to manage, protect, and monitor traffic between microservices, allowing developers to focus on business logic rather than network issues.

The official documentation is as follows:

https://istio.io/latest/en/docs/concepts/traffic-management/

1.1 Installing the Istio Environment

Download the istioctl client tool:

curl -L https://istio.io/downloadIstio | sh -

Add the istioctl client tool to your environment variables:

cd istio-1.21.2/
export PATH=$PWD/bin:$PATH

Test the environment by checking the version of istioctl:

istioctl version

Use istioctl to install the istio environment, provided that the current machine already has a k8s environment.

istioctl install --set profile=demo -y

The available profile options are as follows:

• • default: Starts components based on the default settings of the IstioOperator API. Recommended for production deployments and as the Primary Cluster in Multicluster Mesh. You can run istioctl profile dump to see the default settings.

• • demo: This configuration has moderate resource requirements and is designed to showcase Istio‘s capabilities. It is suitable for running the Bookinfo application and related tasks. This profile enables high-level tracing and access logging, so it is not suitable for performance testing.

• • minimal: Similar to the default profile, but only installs control plane components. It allows you to use Separate Profile to configure control plane and data plane components (such as Gateway).

• • remote: Configures the Multicluster Mesh for the Remote Cluster.

• • empty: Deploys nothing. Can serve as a base profile for custom configurations.

• • preview: The preview file contains experimental features. This is for exploring new features of Istio. Stability, security, and performance are not guaranteed (use at your own risk).

After successful installation, check the Istio pod:

kubectl get pods -n istio-system

If the status is Running, it is normal. You can see that the ingress and egress gateways are installed. These two gateways actually run an Envoy proxy instance at the edge of the mesh as load balancers. The ingress gateway receives incoming connections, while the egress gateway receives connections going out from the cluster.

Create a namespace to operate all subsequent actions within that namespace:

kubectl create ns test

Add a label to this namespace to indicate that the Envoy sidecar proxy should be automatically injected during application deployment:

kubectl label namespace test istio-injection=enabled

1.2 Testing the Istio Environment

In the Istio service mesh, traffic routing rules can be defined through VirtualService, and traffic can be accessed through GateWay. Below is a case to test whether the environment is functioning normally.

Start an Nginx as the target service, creating interfaces /, /buy, /order, and /err for different scenarios:

vi nginx-v1.yml

apiVersion: v1
kind: Service
metadata:
  name: nginx-v1
  namespace: test
  labels:
    app: nginx-v1
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: nginx-v1

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-v1-config
  namespace: test
data:
  nginx.conf: |-
    events {
        use epoll;
        worker_connections  65535;
    }
    http {
        include       mime.types;
        server_tokens off;
        client_max_body_size 50m;
        server {
            listen       80;
            server_name    localhost;
            proxy_buffering off;
            autoindex on;
            add_header Access-Control-Allow-Origin '*';
            proxy_connect_timeout 1800s;
            proxy_send_timeout 1800s;
            proxy_read_timeout 1800s;

            location / {
                return 200 "This is V1 Nginx.";
            }
                
            location /buy {
                return 200 "This is V1 Nginx, Current Uri /buy .";
            }
            
            location /order {
                return 200 "This is V1 Nginx, Current Uri /order .";
            }
            
            location /err {
                return 500 "This is V1 Nginx, Current Uri /err .";
            }
        }
    }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
  namespace: test
spec:
  selector:
    matchLabels:
      app: nginx-v1
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx-v1
        name: nginx
        version: v1
    spec:
      containers:
      - name: nginx-v1
        image: nginx:1.20.1
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/nginx.conf
          subPath: nginx.conf
      volumes:
      - name: nginx-config
        configMap:
          name: nginx-v1-config

kubectl apply -f nginx-v1.yml

Check the Pod:

There are two containers here because one is the sidecar and the other is the nginx target service.

Next, create a Gateway and VirtualService to test traffic forwarding:

vi nginx-gwvs.yml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: nginx-gw
  namespace: test
spec:
  selector:
    istio: ingressgateway  # use istio default controller
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "*"  
  gateways: #bind gateway
    - nginx-gw
  http:
    - route: #route
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80

kubectl apply -f nginx-gwvs.yml

Check the port of the gateway nodeport:

kubectl get svc -n istio-system

Access: http://{node ip}:30868/

Use the following command to check the logs of the ingressgateway:

kubectl logs -f -n istio-system -l istio=ingressgateway

Next, we will start a more detailed usage introduction.

2. VirtualService Traffic Management

Istio‘s VirtualService is a core configuration resource that allows users to control data flow in the service mesh by defining a set of traffic routing rules. These rules can route traffic to different service versions or destinations based on request attributes (such as paths, headers, query parameters, etc.), while supporting features like redirects, rewrites, fault injection, timeouts, retry policies, and traffic mirroring. These features make it simple and flexible to achieve fine-grained traffic management in complex microservices environments.

Below are important configuration items in VirtualService:

Configuration Item	Description
spec.hosts	Defines a set of hosts associated with routing rules, which can be wildcard DNS names or IP addresses.
spec.gateways	Defines the source traffic for applying routing rules, which can be one or more gateways, or sidecars within the mesh, specified as <gateway namespace>/<gateway name>
spec.http.match	Defines a list of matching rules for routing, where all conditions within a single match rule are AND relationships, while multiple match rules in the list are OR relationships.
spec.http.route	Defines a list of forwarding destinations for the HTTP route, which can be a redirect or forwarding (default). The forwarding destination can be one or more services (service versions). It can also configure weights, header operations, etc.
spec.http.redirect	Defines routing redirection; an HTTP route can be either a redirect or forwarding (default).
spec.http.rewrite	Defines rewriting, which cannot be configured simultaneously with redirection; rewrite operations will be executed before forwarding.
spec.http.timeout	Defines the timeout for HTTP requests.
spec.http.retries	Defines the retry policy for HTTP requests.
spec.http.fault	Defines the fault injection policy for HTTP traffic; when enabled, timeout and retry policies will not be active.
spec.http.mirror	Defines the mirroring of HTTP traffic to another specified destination.
spec.http.mirrorPercent	Defines the percentage of traffic to be mirrored; by default, 100% of the traffic is mirrored. The maximum value is 100.
spec.http.corsPolicy	Defines the CORS policy.
spec.http.headers	Defines header operation rules, allowing updates to request and response header, including adding and removing operations.

2.1 Restricting Request Domains

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: nginx-gw
  namespace: test
spec:
  selector:
    istio: ingressgateway  # use istio default controller
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "www.xbc.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways: #bind gateway
    - nginx-gw
  http:
    - route: #route
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80

Update the configuration:

kubectl apply -f nginx-gwvs.yml 

Access again using http://{node ip}:30868/:

At this point, a 404 is returned. Add a domain mapping in hosts:

11.0.1.144 www.xbc.com

Using the domain, access is successful:

2.2 Match Rule Matching

Supports operations on uri, scheme, method, and authority, each type supports exact, prefix, and regex matching modes.

For example:

2.2.1 Routing by URI Prefix

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /buy
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80

When the uri has the prefix buy, it can be accessed normally:

If accessing the order interface, it will not be accessible:

2.2.2 Routing by Header Parameter

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - headers:
          userid:
            exact: test
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80

Only requests with header containing userid=test can access. If the header parameter is not added, a 404 will be returned:

Accessing with the header parameter can be done normally:

2.2.3 Mixing Multiple Rules

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - headers:
          userid:
            exact: test
        uri:
          prefix: /buy
      - headers:
          userid:
            exact: admin
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80
      

The above means that if the header has a value of userid as test and the uri has a prefix of buy, it can be accessed, or if the header has a value of userid as admin, it can also be accessed.

When the uri has a prefix of buy, but the header has a value of userid as anything else, it will return a 404:

When the uri has a prefix of buy, but the header has a value of userid as test, access is normal:

When the uri has a prefix of order and the header has a value of test, it will return a 404:

When the uri has a prefix of order and the header has a value of admin, access is normal:

2.3 Route Traffic Transfer

We have already briefly used route to specify the target service. The route has more functionalities, such as implementing load balancing, gray releases, and traffic transfers, with three main configurations: destination (target request), weight (weight), and headers, where destination is mandatory.

To facilitate testing the target route, deploy another nginx v2 version:

vi nginx-v2.yml

apiVersion: v1
kind: Service
metadata:
  name: nginx-v2
  namespace: test
  labels:
    app: nginx-v2
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: nginx-v2

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-v2-config
  namespace: test
data:
  nginx.conf: |-
    events {
        use epoll;
        worker_connections  65535;
    }
    http {
        include       mime.types;
        server_tokens off;
        client_max_body_size 50m;
        server {
            listen       80;
            server_name    localhost;
            proxy_buffering off;
            autoindex on;
            add_header Access-Control-Allow-Origin '*';
            proxy_connect_timeout 1800s;
            proxy_send_timeout 1800s;
            proxy_read_timeout 1800s;

            location / {
                return 200 "This is V2 Nginx.";
            }
                
            location /buy {
                return 200 "This is V2 Nginx, Current Uri /buy .";
            }
            
            location /order {
                return 200 "This is V2 Nginx, Current Uri /order .";
            }
            
            location /err {
                return 500 "This is V2 Nginx, Current Uri /err .";
            }
        }
    }

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v2
  namespace: test
spec:
  selector:
    matchLabels:
      app: nginx-v2
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx-v2
        name: nginx
        version: v2
    spec:
      containers:
      - name: nginx-v2
        image: nginx:1.20.1
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/nginx.conf
          subPath: nginx.conf
      volumes:
      - name: nginx-config
        configMap:
          name: nginx-v2-config

kubectl apply -f nginx-v2.yml

2.3.1 Weight-Based Traffic Transfer

By configuring the weight parameter, traffic can be allocated proportionally to different target services.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80
        weight: 50
      - destination:
          host: nginx-v2.test.svc.cluster.local
          port:
            number: 80
        weight: 50

The two service versions are allocated 50% of traffic each, and access should demonstrate the load balancing effect:

2.3.2 Adding/Modifying Headers

Headers in request or response can be added or modified:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80
        headers:
          response:
            add:
              test: xbc
            set:
              content-type: application/json

After making a request, you can see the added and modified parameters in the returned header:

2.4 Redirect

You can redirect from one uri to another uri, returning a 302 status code to the service caller, along with the location in the header for the redirected address.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /buy-v2
      redirect:
        uri: /buy
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80    

The above configuration automatically redirects to the /buy interface when accessing /buy-v2:

2.5 Rewrite

Similar to the redirect configuration, but rewrites are not visible to the user and occur on the server side.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /buy-v2
      rewrite:
        uri: /buy
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80    

The above configuration rewrites to the /buy interface when accessing /buy-v2, which is not perceived by the service caller:

2.5 Retries

When a request fails, retry policies can be used to attempt to resend, improving service quality.

This mainly includes three parameters: attempts, perTryTimeout, retryOn:

• • attempts: The number of retries required.

• • perTryTimeout: The timeout for retries (ms, s, m, h).

• • retryOn: Retry policy, multiple are separated by commas; strategies include:

• • 5xx: When the service returns a 5xx status code or does not return.

• • gateway-error: Similar to 5xx exceptions but only for 502, 503, 504.

• • connect-failure: When the connection to the target service fails.

• • retriable-4xx: When the target service returns 4xx.

• • refused-stream: When the target service resets with the REFUSED_STREAM error code.

• • cancelled: When the gRPC response’s Header status code is cancelled.

• • deadline-exceeded: When the gRPC response’s Header status code is deadline-exceeded.

• • internal: When the gRPC response’s Header status code is internal.

• • resource-exhausted: When the gRPC response’s Header status code is resource-exhausted.

• • unavailable: When the gRPC response’s Header status code is unavailable.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80    
      retries:
        attempts: 3
        perTryTimeout: 3s
        retryOn: 5xx,connect-failure

The above defines that when the target service returns 5xx or the connection fails, it will retry a total of 3 times, with a timeout of 3s for each retry.

You can check the logs of nginx-v1 in advance:

kubectl logs -f nginx-v1-dbf758949-bhbdr -n test

Accessing the /err interface once will return a 500 error code:

From the nginx-v1 logs, you can see that it was accessed 4 times, including 3 retries.

2.6 Mirror Traffic

Traffic mirroring allows traffic to be forwarded to the target service while also sending a copy of the traffic to another service, which is very helpful for testing new version environments before going live. The configuration parameters are the same as destination:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80    
      mirror:
        host: nginx-v2.test.svc.cluster.local
        port: 
          number: 80

The above configuration sends requests to nginx v1 while also sending a request to nginx-v2:

You can check the logs of nginx-v2 in advance:

kubectl logs -f nginx-v1-dbf758949-bhbdr -n test

From the logs of nginx-v2, you can see that it was also requested:

2.7 Fault Injection

Fault injection is a testing technique in the service mesh that allows specific faults to be introduced into the system to simulate network delays or service interruptions. The purpose is to test and improve the resilience and stability of the microservices architecture, ensuring that the system can still handle requests correctly when faced with various real-world problems.

Note: Fault injection should be used cautiously, preferably in a testing environment to avoid unnecessary impacts on the production environment.

Fault injection mainly includes two types: delay and abort:

• • delay: Used to simulate network delays or service response delays. By introducing delays, you can test the timeout handling, retry mechanisms, and overall performance of the service or system.

• • abort: Used to simulate service interruptions by returning specific HTTP status codes to simulate service failures. This can be used to test the error handling capabilities, circuit breakers, and other fault recovery mechanisms of the service.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80    
      fault:
        delay: 
          percentage:
            value: 10
          fixedDelay: 5s
        abort:
          percentage:
            value: 10
          httpStatus: 500

The above configuration introduces a 5 second delay for 10% of requests and directly aborts 10% of requests, returning a 500 status code:

2.8 Cross-Origin Configuration

VirtualService provides parameters allowOrigin, allowMethods, allowHeader, exposeHeader, maxAge, allowCredentials to allow flexible cross-origin configuration.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /
      route:
      - destination:
          host: nginx-v1.test.svc.cluster.local
          port:
            number: 80    
      corsPolicy:
        allowOrigin: 
        - www.xbc.com
        allowMethods:
        - "GET"
        - "POST"
        - "PUT"
        - "DELETE"
        maxAge: "24h"

3. DestinationRule Traffic Management Policies

DestinationRule is a configuration resource in the Istio service mesh that defines routing rules when reaching specific services, as well as traffic policies when communicating with that service. DestinationRule is typically used in conjunction with VirtualService, and takes effect when both specify the same hostname and subsets.

Below are important configuration items in DestinationRule:

Configuration Item	Description
spec.host	Associates with destination.host service name.
spec.subsets	Service subsets.
spec.trafficPolicy	Traffic policies, including load balancing, connection pools, health checks, and TLS policies.
spec.trafficPolicy.loadBalancer	Load balancing algorithms, supporting round robin, least connections, random, etc.
spec.trafficPolicy.connectionPool	Connection pool configurations.
spec.trafficPolicy.outlierDetection	Circuit breaker configurations.

Next, we will keep the configuration of Gateway and VirtualService specified as follows, focusing on configuring the information of DestinationRule:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: nginx-gw
  namespace: test
spec:
  selector:
    istio: ingressgateway  # use istio default controller
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "www.xbc.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"
  gateways: 
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /
      route:
      - destination:
          host: nginx.test.svc.cluster.local
          port:
            number: 80

3.1 Traffic Transfer – Load Balancing

In k8s, Service itself also provides a load balancing mechanism, while DestinationRule offers more advanced load balancing features and traffic control methods, allowing traffic to be divided among multiple versions of a service, such as round robin, least connections, random, etc., and also allowing different weights of traffic to be transferred to different versions.

Next, create a Service that points to both nginx-v1 and nginx-v2 Pods:

vi nginx-svc.yml

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: test
  labels:
    app: nginx
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    name: nginx

kubectl apply -f nginx-svc.yml

Load balancing configurations have two main options: simple (simple load balancing) and consistentHash (consistent Hash algorithm):

simple includes:

• • ROUND_ROBIN: Round robin, the default strategy.

• • LEAST_CONN: Selects a target service with relatively fewer active requests.

• • RANDOM: Randomly selects one.

• • PASSTHROUGH: Directly forwards to the target service without any strategy.

consistentHash includes:

• • httpHeaderName: Calculates the hash based on the header.

• • httpCookie: Calculates the hash based on cookies.

• • useSourceIp: Calculates the hash based on the IP address.

• • minimumRingSize: Minimum number of virtual nodes on the hash ring; the more nodes, the more balanced the distribution.

Create a new DestinationRule configuration:

vi nginx-dr.yml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: nginx-dr
  namespace: test
spec:
  host: nginx.test.svc.cluster.local
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN #Load balancing algorithm
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

kubectl apply -f nginx-dr.yml

Through testing, you can feel the effect of load balancing:

3.2 Traffic Transfer – Transferring Traffic Between Different Versions of the Same Service

In the VirtualService, we introduced weight-based traffic distribution, but for different hosts, combining with DestinationRule allows for more fine-grained control of distribution among multiple versions within a single host, such as controlling 80% of traffic to v1 and 20% to v2.

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: nginx-dr
  namespace: test
spec:
  host: nginx.test.svc.cluster.local
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-vs
  namespace: test
spec:
  hosts:
    - "www.xbc.com"  
  gateways:
    - nginx-gw
  http:
    - match:
      - uri:
          prefix: /
      route:
      - destination:
          host: nginx.test.svc.cluster.local
          port:
            number: 80
          subset: v1
        weight: 80
      - destination:
          host: nginx.test.svc.cluster.local
          port:
            number: 80
          subset: v2
        weight: 20

You can loop 10 requests in the console:

for /l %i in (1,1,10) do curl http://www.xbc.com:30868/

The results will show that the v1 service received 80% of the traffic.

3.3 Connection Pool

The connection pool can be used to control the number of connections sent to the target service, including both TCP connections and HTTP connections.

TCP connection parameters:

• • maxConnections: Maximum number of connections, default is 1024, also applies to HTTP/1.1; HTTP/2 will use a single connection for each host.

• • connectTimeout: Connection timeout.

• • tcpKeepalive: A new configuration supported after Istio1.1 that periodically sends a keepalive probe to the target service to check if the connection is available.

HTTP connection parameters:

• • http1MaxPendingRequests: Maximum number of pending requests, default is 1024, only applies to HTTP/1.1 services.

• • http2MaxRequests: Maximum number of requests, default is 1024. Only applies to HTTP/2 services.

• • maxRequestsPerConnection: Maximum number of requests per connection. Both HTTP/1.1 and HTTP/2 adhere to this parameter. If not set, it indicates no limit. Setting it to 1 means that each connection only handles one request, effectively disabling Keep-alive.

• • maxRetries: Maximum number of retries, default is 3.

• • idleTimeout: Idle timeout; if there is no activity within this time, the request will close the connection.

trafficPolicy controls can be specific to a subset:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: nginx-dr
  namespace: test
spec:
  host: nginx.test.svc.cluster.local
  trafficPolicy: 
    connectionPool: 
      tcp:
        maxConnections: 1024
        connectTimeout: 30s
        tcpKeepalive:
          probes: 5
          time: "3600"
          interval: 60s
      http: 
        http1MaxPendingRequests: 2048
        http2MaxRequests: 2048
        maxRequestsPerConnection: 10
        maxRetries: 3
        idleTimeout: 60s
  subsets:
  - name: v1
    labels:
      version: v1
    trafficPolicy: 
      connectionPool: 
        http: 
          http1MaxPendingRequests: 1024
          http2MaxRequests: 1024
          maxRequestsPerConnection: 10
          maxRetries: 3
          idleTimeout: 60s
  - name: v2
    labels:
      version: v2
    trafficPolicy: 
      connectionPool: 
        http: 
          http1MaxPendingRequests: 1024
          http2MaxRequests: 1024
          maxRequestsPerConnection: 10
          maxRetries: 3
          idleTimeout: 60s
  

3.2 Circuit Breaker

The circuit breaker is a protection mechanism in the Istio service mesh, used to prevent system avalanches and ensure service stability. The Istio circuit breaker mechanism monitors services in real-time, and when it detects that a service has continuous failures, excessive latency, or abnormal responses, it automatically triggers the circuit breaker to prevent further requests from being sent to that service. This can avoid invalid requests from occupying system resources while giving the faulty service time and space to recover.

The circuit breaker functionality in Istio is primarily implemented through the outlierDetection configuration item in DestinationRule, which includes the following configurations:

• • consecutiveErrors: Indicates that if a service instance returns errors consecutively for a certain number of times within the interval time, that instance will be considered abnormal and trigger the circuit breaker. The default value is 5.

• • interval: Defines the statistical time window for consecutiveErrors. Within this time window, if the failure count of the service instance reaches the threshold of consecutiveErrors, the instance will be marked as abnormal. The default value is 10 seconds.

• • baseEjectionTime: Indicates the minimum time for an abnormal instance to be ejected. Once an instance is ejected, it must wait at least this time before it can rejoin the load balancing pool, even if it becomes available. The default value is 30 seconds.

• • maxEjectionPercent: Limits the maximum percentage of instances that can be ejected at the same time. This configuration prevents all instances from being ejected simultaneously, ensuring that the service has a certain level of availability. The default value is 10%.

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: nginx-dr
  namespace: test
spec:
  host: nginx.test.svc.cluster.local
  trafficPolicy: 
    connectionPool: 
      tcp:
        maxConnections: 100
      http: 
        http1MaxPendingRequests: 100
        http2MaxRequests: 100
    outlierDetection: 
      consecutiveErrors: 5
      interval: 1m
      baseEjectionTime: 1m
      maxEjectionPercent: 100
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

The above configuration indicates that if an error occurs 5 times within 1 minute, the circuit will be triggered, with a duration of 1 minute, allowing for the ejection of all services.

Next, use the console to concurrently access the /err interface 20 times:

for /l %i in (1,1,20) do start  curl http://www.xbc.com:30868/err

After executing, test accessing the /err interface through PostMan:

If you access other interfaces, /buy will also be inaccessible because the entire target service has been ejected:

After waiting for 1 minute, accessing again will return to normal: