The Risks of Using eBPF for Security

eBPF (extended Berkeley Packet Filters) has emerged as a powerful tool for extending the functionality of the Linux kernel, enabling the implementation of security solutions. However, using eBPF for security purposes is not without risks. This paper explores the vulnerabilities and challenges associated with leveraging eBPF as part of a security solution on Linux. In particular, memory constraints, the eBPF verifier, lack of anti-tampering capability, and external data access are discussed. In fact, Linux security experts behind the development of security modules such as SELinux and AppArmor at the 2023 North American Linux Security Summit who describe eBPF as “terrifying” from a security standpoint.

Why eBPF?

Originally designed for network packet filtering, eBPF evolved to become a general-purpose framework for extending kernel functionality. It enables the creation and execution of custom programs directly in the kernel, offering benefits such as improved performance and reduced context switching overhead compared to traditional user-space solutions.

Taking advantage of eBPF can be very convenient for security applications. It provides the ability to monitor and analyze network traffic in real-time, making it possible to implement fine-grained filtering and intrusion detection mechanisms. It also allows for the tracing of system events such as file operations, system calls and memory management. This capability can enable rapid identification of anomalies and enforcement of security policies.

For many Linux environment maintainers, security solutions which leverage eBPF can seem like the best of all worlds: increased fidelity, stability, and speed. A well implemented eBPF security solution may realize these advantages when compared with more traditional legacy software

Why Not eBPF?

Unfortunately, all the benefits are not without several potential concerns when employing eBPF for security purposes. Below, we will address the risks of memory constraints, bugs in the bytecode verifier, added attack surfaces for malware injection into the kernel, lack of anti-tampering implementation, and inability to proactively prevent actions. This is not meant as a wholesale condemnation of eBPF as a technology, but an aid in gaining an understanding of the significant risks imposed by relying on eBPF for security.

Memory Constraints

eBPF programs are constrained by the amount of memory they may use. Typically, all data acquired from eBPF instrumentation is stored in one of several predefined eBPF memory map data structures. To keep resource usage low, these structures are typically very small. Even in purpose-built mapping schemes such as ring buffers, the rate at which eBPF programs generate data may exceed available memory and exhaust resources. There is a vulnerability found in Falco (CVE-2019-8339) which proves that malicious system calls can be routinely missed by eBPF monitoring if memory resources are exhausted with benign data.

To take advantage of CVE-2019-8339, one only needs to call benign, monitored, system calls repeatedly in quick succession over a short time. This behavior may fill the memory maps used to report potential security events. If a malicious action is taken while the maps are at or near capacity, the malicious action is dropped. This behavior is not difficult to create and can be executed by an unprivileged user. This specific vulnerability is representative of the general weakness associated with limited memory resources in eBPF software.

Verifier Bugs

It is no secret that giving kernel-level executable ability to user-space software can be dangerous. This danger is why the eBPF subsystem includes protections and verifications to ensure that no undue access is granted to underprivileged users or software. The most crucial part of these protections is the eBPF verifier. The verifier performs a series of checks on eBPF bytecode before allowing it to be passed to the JIT compiler and eventually executed in the kernel. These checks include assurance that the code will terminate within a set time limit, that variable types are used in consistent ways throughout the code, and that the overall complexity of the code is within limits. In this way, the verifier acts as a “gatekeeper” to the kernel. Once the code passes the verifier, there is nothing stopping it from executing with kernel-level permission.

 

The issue with BPF-based software is not a lack of privilege, but rather unfettered access to kernel-level action. For many attackers, vulnerabilities in eBPF are a very desirable vector toward privilege escalation. In the last 2 years (since 2021), at least 9 significant CVE’s can be identified dealing with eBPF and verifier/input validation issues. These CVEs include 2022-23222 a high severity privilege escalation and 2021-4159 which can leak kernel memory details.

Self-Protection

One of the most glaring issues with using eBPF for critical security is the lack of anti-tamper capabilities. eBPF provides read-only access to system call parameters and is non-blocking. Thus, security applications are only able to see an event after it has occurred. This detail enables attackers to freely execute actions such as killing a security process. Unfortunately, this same attribute makes eBPF programs easy to manipulate, start, and kill from a privileged account for attackers as well as defenders. If an attacker gains privileged access to a system via vulnerability exploitation, privilege escalation, or credential stealing, they will then be able to completely disable any security software centered on eBPF.

Although simply killing eBPF processes may be effective, it is also particularly “noisy” and will not go unnoticed by system administrators. Unfortunately, the same issue of self-protection applies to the memory maps eBPF uses to communicate with user space processes. Any privileged account will have access to memory maps and will be free to read, write or modify them at will. This severely hampers the integrity of data gained from eBPF programs and introduces a host of attack and obfuscation vectors for attackers.

More Attack Surface/External Access

Attackers have taken advantage of eBPF’s kernel abilities. Perhaps the most famous is the malware campaign, “BPFdoor.” BPFdoor uses BPF instrumentation to trigger receipt of network packets before they reach the machine’s internal firewall. The malware is then able to completely circumnavigate the firewall, opening new connections between the victim and the attacker despite strong network protections.

BPFdoor is obviously not a security product. However, the ability it demonstrates is concerning. Security products using eBPF will always be at risk not only of interacting with network/remote resources in unexpected ways, but they also open themselves to exploitation via less visible ingress points. The increase of the attack surface does not apply only to networked communications, but also to actions taken within user-space. User input and unprivileged code heavily monitored by eBPF could require rigorous sanitization of the retrieved data.

Writing secure software is an industry wide struggle, especially with new or expanded technologies. When security products are written around less-mature features, such as eBPF, there are bound to be growing pains. If attackers continue to show sophisticated use of eBPF as seen with BPFdoor, relying on eBPF for detection or remediation of security concerns may not be the wisest action.

Reporting, not Remediation

Instrumentation and diagnostics are the primary focuses of eBPF, and remediation of malicious action is not. Kernel-level observation and reporting are relatively easy in eBPF-based software, but robust control over modifications made by malicious software is a trickier issue. eBPF is nonblocking, meaning it cannot pause or prevent an action from completing. For example, if an attacker deletes a file with an “unlink” system call, eBPF can notify about this action only after the file is gone. This issue makes the response capabilities of eBPF-based solutions extremely weak.

In modern security there is a cliché which states that the question of system compromise is not a question of “if” but “when.” When compromise happens eBPF may be able to quickly alert you to the issue, but it will not be able to reverse malicious changes such as encrypted files and drives, modified credentials, or corrupted configurations on its own.

Pairing an eBPF-based solution alongside more robust filesystem backup/remediation schemes is a possible solution but presents its own challenges. The increased reliance on backups amplifies the traditional problems they have. The business expense of the time to recover can be significant depending on the restoration scheme, and the reliability of those backups is paramount. This increased complexity does not provide redundancy with an eBPF solution but rather supplements a deficiency of it. An endpoint solution with native file tampering detection, prevention and remediation is a more desirable choice.

In the latest data breach report released by IBM Security, AI or automated technologies on average saved breached organizations $3.05 million. These solutions augmented or replaced human intervention. The nonblocking nature of eBPF makes it ill-suited to intervene and provide the benefit obtained from automated security, leaving the notable advantage found in IBM’s reporting unrealized.

Performance Issues

eBPF is touted as being highly performant. In comparison to other kernel instrumentation such as auditd, this quality is generally correct. However, there are situations where eBPF can have significant performance impact on systems. See Vali Cyber’s CTO, Austin Gadient, speak about performance measurement for security programs at the 2023 Linux Security Summit North America. In the talk, he discusses the performance overhead of an eBPF security agent, Falco. The details depend on the implementation of the eBPF program and the user space application that accesses data produced by eBPF. eBPF security tools can be measured effectively using benchmarking tools such as SecurityPerf.

Conclusion

eBPF is a powerful tool for monitoring system events. It can offer much to future developers in low-latency system instrumentation. However, its advantages should not be taken without weighing the relative risks presented by resource exhaustion, lack of self-protection, and verifier flaws, among others. eBPF has undergone continuous improvement and bug fixes to address security concerns as they have been discovered. Staying up to date with the latest patches and following security best practices is crucial to mitigate these vulnerabilities and is especially critical while using eBPF to protect critical operations.