How Open Source Project Tetragon Is Evolving Security via eBPF

Over the last decade, the eBPF open-source project quietly laid the groundwork for major evolutionary gains in Linux subsystems and how they keep pace with the new world of microservices and distributed applications. Today, that foundation has made possible eBPF "programs" that bring new approaches to classic challenges in distributed systems. One of the most interesting examples of an eBPF program with a lot of momentum is Tetragon — the open-source project tackling some of the run-time security's trickiest requirements for developers and platform engineers. I interviewed Jeremy Colvin, senior engineer at Isovalent, to learn more.

Q&A With Jeremy Colvin

Q:  How did eBPF lay the groundwork for programs like Cilium and Tetragon, and why is the ability to add programs to the kernel (without modifying the kernel) kickstarting so many interesting new programs?

A: eBPF (extended Berkeley Packet Filter) has revolutionized the way programs interact with the Linux kernel by allowing the execution of advanced sandboxed programs directly in the kernel without the need to modify the kernel source code or load custom kernel modules.

This capability is a game-changer because it enables the addition of new functionalities with minimal performance overhead and enhanced security directly inline. Now, teams can quickly write policies to monitor certain events, observe closer to the kernel, and take direct enforcement action faster than ever.

For Cilium and Tetragon, eBPF provides a powerful foundation for implementing advanced networking and security features, such as fine-grained observability, real-time enforcement, and direct data collection/aggregation within the kernel. eBPF’s flexibility and efficiency have spurred a wave of innovation, as platform and security teams can quickly build complex, high-performance policies that sit in kernel space, opening up new possibilities for monitoring, security, and networking applications.

Q:  What was the vision for Tetragon? Why was it created, and what are its key primitives?

A: The core principles of Tetragon remain: to create an open-source eBPF security tool that offers extremely deep observability at low overhead, with the identity-awareness that cloud-native workloads demand, to make it easy for teams to deploy and get value.

Tetragon has succeeded in becoming the flexible, Kubernetes-aware security observability and runtime enforcement tool that leverages unique expertise in eBPF for reduced observation overhead and real-time policy enforcement.

The key primitives of Tetragon include high-level declarative policies, fine-grained eBPF-powered observability, and enforcement capabilities for Kubernetes and other Linux workloads. These principles allow Tetragon to deliver deep visibility into system activities, enforce security policies at various levels (such as namespaces, nodes, and pods), and integrate seamlessly with existing security and observability tools, addressing the complex security challenges faced by modern cloud-native environments.

Q: How do you see the security discipline changing? What are some of the pressures on security engineering teams that have driven this new wave of open source, particularly in run-time security?

A: The security field is evolving rapidly with the increasing complexity and scale of modern IT environments, particularly with the rise of cloud-native technologies and microservices architectures. Security engineering teams are pressured to provide deep and wide security measures while maintaining high performance and minimizing operational overhead.

This has driven a renewed interest in open-source solutions, particularly runtime security, as organizations seek more flexible, transparent, and community-driven tools to address their security needs.

Key pressures include the need for real-time threat detection and response for ephemeral systems (containers, pods, VMs, etc.), comprehensive observability across distributed systems, and the ability to enforce security policies at scale. Some teams may struggle with complex threat detection rules, while for others, it’s as foundational as getting the best visibility into their production environment.

Q: When you consider software supply chain security vulnerabilities like Log4 and XZ Utils, what are the new types of security observability and policy enforcement scenarios that teams are trying to improve, and where does Tetragon fit?

A: In the context of software supply chain security vulnerabilities like Log4j and XZ Utils, security teams focus on improving their capabilities for quickly detecting and responding to new threats in real-time.

An extension of these branches off two ways: first, spinning out detection policies with Tetragon in minutes, and second, adequately patching your environment as new CVEs emerge. Teams are looking for enhanced observability in executing packages or binaries, more effective synchronous policy enforcement mechanisms, and traceability of what processes are accessing critical resources.

Tetragon has SecOps and platform teams excited by providing deep visibility into process execution, file access, and network activity. The real-time enforcement capabilities with eBPF allow teams to block malicious actions at the kernel level, preventing potential exploitation or TOCTOU vulnerabilities. Also, Tetragon’s ability to correlate network and runtime events paints a clearer picture of activities across runtime.  

Q:  How does enforcement or alerting work in Tetragon? How does Tetragon use eBPF to apply stronger enforcement policies than legacy tools?

A: Traditionally, the core decision logic of security tools sits with the agent in userspace. This means that security events are sent to the agent and evaluated against certain policy actions; ie, if this violates a policy, take XYZ action. But what teams have quickly discovered is this creates a delay in the policy enforcement point. Instead of observing and taking action in one motion (ideally in the kernel where the event is occurring/before it can execute), traditional EDR and observability tools observe events -> move them to userspace -> then take a policy decision. This is too late to be effective.

Tetragon’s in-kernel enforcement policies are stronger than traditional reactive policies, as the decision logic and action are both in the kernel. These in-kernel enforcement policies offer a more robust approach to security by blocking malicious actions before they occur, providing a proactive security model.

• Enforcement: It will block the kernel's action and prevent it from happening.
• Reactive Security (alerting): Allows user space to react to events and then take action.

Tetragon supports both enforcement and reactive alerting models. For example, Sigkill and Override (enforcement) vs. Post, Alert (alerting).

Tetragon balances these models by offering both enforcement (e.g., blocking actions like file writes or socket sends in the kernel) and reactive alerting (e.g., generating alerts post-event for further analysis). This approach gives security teams the flexibility to block threats proactively while also being able to log and analyze security-significant events reactively.

Q:  Network and run-time observability have typically been two disparate things for security engineers to try to correlate in troubleshooting. How is Tetragon making it possible to see these two things together?

A: One of the most valuable aspects of Tetragon’s observability is bringing together the network and runtime layers. You can monitor and correlate network traffic (TCP/UDP/DNS/HTTP traffic) and runtime activities (the binaries and processes that launched that traffic).

For example, Tetragon traces network connections back to the specific binaries that spawned them, providing a clear link between network traffic and the processes (and even parent processes) responsible for it. This is crucial for identifying the source of suspicious network activity, as it allows security engineers to see what network events are happening and which applications or processes are generating them.

By tying network events directly to process execution, file access, and other runtime behaviors, Tetragon offers a comprehensive view of system activity. This simplifies troubleshooting by providing a complete picture of network and runtime events in a single platform.

Security engineers use Tetragon to quickly detect and gather all the necessary context to understand the behavior of both network and application layers. This unified visibility opens up new possibilities for detecting anomalies, enforcing security policies, and responding to threats more accurately and quickly.