What Is a Service Mesh and Why Is It Needed for Kubernetes?

Enterprises nowadays are keen on adopting a microservices architecture, given its agility and flexibility. Containers and the rise of Kubernetes — the go-to container orchestration tool — made the transformation from monolith to microservices easier for them.

However, a new set of challenges emerged while using microservices architecture at scale:

• It became hard for DevOps and architects to manage traffic between services
• As microservices are deployed into multiple clusters and clouds, data goes out of the (firewall) perimeter and is vulnerable; security becomes a big issue
• Getting overall visibility into the network topology became a nightmare for SREs.

Implementing new security tools, or tuning existing API gateway or Ingress controllers, is just a patchwork and not a complete solution to solve the above problems.

What architects need is a radical implementation of their infrastructure to deal with their growing network, security, and observability challenges. And that is where the concept of service mesh comes in.

What Is a Service Mesh?

A service mesh decouples the communication between services from the application layer to the infrastructure layer. The abstraction at the infrastructure level happens by proxying the traffic between services (see Fig. A).

Fig A — Service-to-service communication before and after service mesh implementation

The proxy is deployed alongside the application as a sidecar container. The traffic that goes in and out of the service is intercepted by the proxy, and it provides advanced traffic management and security features. On top of it, service mesh provides observability into the overall network topology.

In a service mesh architecture, the mesh of proxies is called the data plane, and the controller responsible for configuring and managing the data plane proxies is called the control plane.

Why Do You Need a Service Mesh for Kubernetes?

While starting off, most DevOps only have a handful of services to deal with. As the applications scale and the number of services increases, managing the network and security becomes complex.

Tedious Security Compliance

Applications deployed in multiple clusters from different cloud vendors talk to each other over the network. It is essential for such traffic to comply with certain standards to keep out intruders and to ensure secure communication.

The problem is that security policies are typically cluster-local and do not work across cluster boundaries. This points to a need for a solution that can enforce consistent security policies across clusters.

Chaotic Network Management

DevOps engineers would often need to control the traffic flow to services — to perform canary deployments, for example. And they also would want to test the resiliency and reliability of the system by injecting faults and implementing circuit breakers.

Achieving such kinds of granular controls over the network requires DevOps engineers to create a lot of configurations and scripting in Kubernetes and the cloud environment.

Lack of Visualization Over the Network

With applications distributed over a network and communications happening between them, it becomes hard for SREs to keep track of the health and performance of the network infrastructure. This severely impedes their ability to identify and troubleshoot network issues.

Implementing service solves the above problems by providing features that make managing applications deployed to Kubernetes painless.

Key Features of Service Mesh in Kubernetes

Service mesh acts as a centralized platform for networking, security, and observability for microservices deployed into Kubernetes.

Centralized Security

With a service mesh, security compliance is easier to achieve as it can be done from a central plane instead of configuring it per service. A service mesh platform can enforce consistent security policies that work across cluster boundaries.

Service mesh provides granular authentication, authorization, and access control for applications in the mesh.

• Authentication: mTLS implementation, JWT
• Authorization: Policies can be set to allow, deny, or perform custom actions against an incoming request
• Access control: RBAC policies that can be set on method, service, and namespace levels

Advanced Networking and Resilience Testing

Service mesh provides granular control over the traffic flow between services. DevOps engineers can split traffic between services or route them based on certain weights.

Besides, service mesh provides the following features to test the resiliency of the infrastructure with little work:

• Fault injection
• Timeouts
• Retries
• Circuit breaking
• Mirroring 

Unified Observability

Implementing service mesh helps SREs and Ops teams to have centralized visibility into the health and performance of applications within the mesh. Service mesh provides the following telemetry for observability and real-time visibility:

• Metrics: To monitor performance and see latency, traffic, errors, and saturation.
• Distributed tracing: To understand requests’ lifecycle and analyze service dependency and traffic flow.
• Access logs: To audit service behavior.

Top Service Mesh Software To Consider for Kubernetes

One may find various service mesh software such as Istio, Linkerd, HashiCorp Consul, Kong KUMA, Google Anthos (built on Istio), VMware Tanzu, etc., in the market.

However, over 90% of the users either use Istio or Linkerd service mesh software because of their strong and vibrant open-source ecosystem for innovation and support.

Istio

Istio is the most popular, CNCF-graduated open-source service mesh software available. It uses Envoy proxy as sidecars, while the control plane is used to manage and configure them (see Fig.B).

Fig B — Istio sidecar architecture

Istio provides networking, security, and observability features for applications at scale. Developers from Google, Microsoft, IBM, and others actively contribute to the Istio project.

Linkerd

Linkerd is a lightweight, open-source service mesh software developed by Buoyant. It provides the basic features of a service mesh and has a destination service, identity service, and proxy injector (see Fig.C).

Fig C — Linkerd architecture

More than 80% of the contributions to Linkerd are by the founders, Buoyant, itself.

(To see a detailed comparison between the two and choose one for your Kubernetes deployments, head to Istio vs Linkerd: The Best Service Mesh for 2023.)

Benefits of Service Mesh for Kubernetes

Below are some benefits enterprises would reap by implementing service mesh in Kubernetes.

100% Network and Data Security

Service mesh helps in maintaining a zero-trust network where requests are constantly authenticated and authorized before processing. DevOps engineers can implement features such as mTLS, which works cluster-wide and across cluster boundaries (see Fig.D).

Fig D — Zero trust network implementation

A zero-trust network helps in maintaining a secure infrastructure in the current dynamic threat landscape filled with attacks, like man-in-the-middle (MITM) and denial of service (DoS).

(If you are interested to learn more, check out this article: Zero Trust Network for Microservices with Istio.)

80% Reduction in Change Failure Rate

Nowadays, enterprises release applications to a small set of live users before a complete rollout. It helps DevOps and SREs to analyze the application performance, identify any bugs, and thus avoid potential downtime.

Canary and blue/green deployments are two such deployment strategies. The fine-grained traffic controls — including splitting traffic based on weights (see Fig.F) — provided by service mesh make it easier for DevOps engineers to perform them.

Fig F — Canary deployment with Istio service mesh

99.99% Available and Resilient Infrastructure

The telemetry data provided by service mesh software helps SREs and Ops teams to identify and respond to bugs/threats quickly.

Most service mesh software integrates with monitoring tools like Prometheus, Grafana, Kiali, Jaeger, etc., and the dashboard (see Fig.G) provided by them helps operators visualize the health, performance, and behavior of the services.

Fig G — Kiali service graph

5X Improvement in Developer Experience

Most application developers do not enjoy configuring the network and security logic in their applications. Their focus tends to be on business logic and building features.

Implementing service mesh reduces developer toil as they are left alone with the application code. They can offload network and security configurations completely to the service mesh at the infrastructure level. The separation helps developers focus on their core responsibilities, i.e., delivering the business logic.

Three Pillars for Successful Implementation of Service Mesh

Since service mesh is a radical concept, it can be overwhelming for enterprises to implement and realize its value successfully. If you are an architect or CIO, you would want to consider the following three pillars for successful service mesh implementation.

1. Technology Support

It is important to evaluate a service mesh software from the technology support perspective. If you are a mature DevOps organization using various open source and open standards in your CI/CD process, ensure that service mesh software integrates well with your CI/CD tools (of whatever versions).

For example, if you are using Argo CD for GitOps deployment, or Prometheus for monitoring, then a service mesh software must be able to integrate with less intervention.

2. Enterprise Support

Open-source software adoption is on the rise. But the support for software will be a prime necessity for enterprises to make sure their IT is available for the business. 

Evaluate a service mesh software that is backed by a large community member (very good for support), and also, there are 3rd party vendor ecosystems that can provide 24*7 support with fixed SLA.

3. Training and Onboarding Support

Ensure there are adequate reading materials, documents, and videos available, which will supplement the learning of users of service mesh software because it would not make sense if internal employees such as DevOps and SREs are not able to adopt it.

Finally, a service mesh is not just software but is an application operation pattern. Do not hasten your project. Rather, research and evaluate the best service mesh that suits your organizational requirements and needs.

Service Mesh Is the Way Forward for Kubernetes Workloads

The goal of service mesh implementation is to make managing applications deployed in Kubernetes easier. Adopting it will become a necessity as services start to overflow from a single cluster to multiple clusters. With Kubernetes adoption on the rise, service mesh will eventually become a critical component in most organizations.