Extending Applications Safely and Efficiently

This paper is included in the Proceedings of the 19th USENIX Symposium on Operating Systems Design and Implementation.</h4>

July 7–9, 2025 • Boston, MA, USA

### 1 Introduction Developer defines new logic for the program as a set of extensions, without modifying the logic of main program, and associates each extension with a specific location in the host application, called the *extension entry*. When the user invokes the application, the system loads the host application and the user's configured extensions. Each time an application thread reaches an extension entry, the thread jumps to the associated extension. It executes extension in the extension runtime context; once the extension completes, the thread returns to the host at the point immediately after the extension entry. This paper presents a new approach for specifying the interface between an extension and a host, called the *Extension Interface Model (EIM)*, and a new extension runtime system, called bpftime, that together provide safer and more efficient software extensions. EIM and bpftime are motivated by the challenges current extension frameworks face in balancing key extensibility features. **Tradeoff** - Interconnectedness – extensions need to be able to observe or modify the host application's state and to execute functions defined by the host. - Safety – host should limit both the system resources that extensions consume, as well as limit the host interactions they perform for safety. - Interconnectedness $$\not\propto$$ Safety, thus need to find a balance while meeting both host and extension requirements. Human *extension manager* is supposed to configure extensions and navigate the tradeoff. Finding such optimal tradeoff while respecting needs is a challenge not properly met so far. Few frameworks (e.g., Orbit, Wedge) do support fine-grained interconnected/safety tradeoffs, but they are not designed for extensibility and require modifying host application source code to impose different tradeoffs. **First contribution: Extension Interface Model (EIM)** - EIM represents extension features through an abstraction called *resource*. - EIM represents the ability to use resources with *capabilities*. - Two steps to produce EIM specification: - First, developers define resources that host application can provide to extensions. - Then extension manager creates extension classes that specify the set of capabilities allowed at a particular extension entry, essentially choosing the tradeoff for each extension entry. - EIM specifications are runtime-agnostic, and an existing extension framework can be enhanced to support them. **Second contribution: bpftime** - A new extension runtime that efficiently supports EIM and extension isolation using two design principles: - First, the system uses lightweight approaches to provide extension safety and isolation. It enforces the safety in an EIM specification without any runtime overhead using **eBPF-style verification**, and it enforces isolation with minimal overhead using ERIM-style intraprocess hardware-supported isolation. - Second, bpftime introduces concealed extension entries, which improve efficiency by eliminating runtime overhead from extension entries that are not in use by a running process. Concealed extension entries use binary rewriting to inject an extension entry into a host only when a user loads an associated extension. While prior work uses techniques similar to bpftime’s verification, isolation, and rewriting techniques, bpftime is the first system to combine them to satisfy EIM’s requirements. - bpftime is fully **compatible with eBPF**, which means not only can current users of eBPF extensions (e.g., uprobes) seamlessly adopt bpftime, but bpftime extensions can also share state and interact closely with eBPF kernel extensions, thereby supporting extensibility use cases that require extending **both the kernel and applications**. - Open-source with active PRs and over 1.4k stars (https://github.com/eunomia-bpf/bpftime) Inspired by the use cases of their users, they present 6 use cases that highlight the benefits of bpftime and EIM: 1. Monitoring a microservice application 2. Creating new durability configuration in Redis 3. Caching metadata operations in FUSE 4. Implementing an SSL-supporting distributed tracing tool 5. Monitoring system calls for performance analysis 6. Enhancing webserver security They evaluate bpftime's performance on those 6 use cases, and discover the following speedups: - bpftime improves the throughput of profiling microservices by a factor of 1.5 compared to eBPF - bpftime enables a Redis durability configuration that loses orders of magnitude less data in a crash while decreasing throughput by only about 10%. - bpftime enables FUSE caching that accelerates operations by orders of magnitude. - The system adds only 2% overhead when extending Nginx, which is up to 5× lower than state-of-the-art alternatives such as WebAssembly, Lua, ERIM, and RLBox. - bpftime reduces the overhead of SSL traffic monitoring by a factor of 3.79 compared to native eBPF. Moreover, bpftime offers configurations that prevent monitoring overhead from affecting unmonitored processes, a feature eBPF cannot provide. In sum, the contributions are: - EIM, which allows users to specify fine-grained interconnectedness/safety tradeoffs. - bpftime, an efficient extension runtime system that enforces EIM specifications and isolation through two separate verfication techniques and concealed extension entries. - An evaluation of EIM and bpftime demonstrating their usefulness and efficiency in 6 use cases. ### 2 Motivation #### 2.1 Roles - Application developers – a group of trusted but fallible developers who write the original application. - Extension developers – a group of trusted but fallible developers who create the extensions. - Extension manager – a trusted and infallible individual that installs and manages the extensions. - Users – untrusted individuals who interact with the extended application; Application and extension developers are considered to be fallible, so applications and extensions might be exploitable but are not intentionally malicious. Users can be malicious and may try to craft inputs that would trigger vulnerabilities in otherwise benign code. 

Figure 1: Human roles involved in the framework

#### 2.2 Web-Server example (Nginx reverse proxy) They use Nginx as a motivating scenario: an extension manager picks and configures multiple extensions—e.g., (1) monitoring for reliability issues, (2) a firewall that blocks suspicious URLs (SQLi/XSS signals), and (3) a load balancer that probes downstream servers and routes based on load. #### 2.3 Key extension-framework features They argue extension frameworks need three things: - **Fine-grained safety vs. interconnectedness tradeoffs**: extensions must read/write host state and call host functions, but the manager should grant only what each extension needs (least privilege), potentially varying per entry and per use case. - **Isolation**: the host shouldn’t be able to tamper with extension state; otherwise attackers could bypass security-monitoring extensions by exploiting the host. - **Efficiency**: extensions may run on hot paths (e.g., load balancing per request), so overhead must be near-native.  #### 2.4 Limitations of state-of-the-art They survey existing approaches and what they miss: - **Native-in-process techniques** (e.g., LD_PRELOAD / DBI): fast, but don’t provide isolation or fine-grained safety tradeoffs. - **SFI-based sandboxes** (Wasm, Lua, NaCl, RLBox, XFI): often either don’t offer a real safety-tradeoff interface (pushing safety checks into the app) or aren’t fine-grained enough; plus SFI tends to be slower due to runtime checks. - **Subprocess isolation** (Wedge, Shreds, lwC, Orbit): good isolation, but either lack fine-grained tradeoffs or require host code changes; also costly to switch between host and extension frequently. - **eBPF uprobes**: isolated, but no fine-grained tradeoffs and inefficient because each entry hit traps into the kernel via a breakpoint. - **Aspect-oriented languages**: allow extensions but don’t provide these safety/tradeoff controls. #### 2.5 Threat model They assume the **extension manager is correct** in choosing the allowed interface (so a buggy extension can’t crash/hang/corrupt *through what’s allowed*), and they assume the app’s **control-flow can’t be hijacked** (CFI-like), otherwise an attacker could skip/bypass extension execution. Within those assumptions, they focus on two threats: (1) **buggy extensions** that crash/hang via things like bad pointers/infinite loops/stack corruption outside what’s allowed, and (2) **compromised applications** that tamper with extension state to change extension behavior. ### 3 Extension Interface Model (EIM) **An EIM specification has two components (two authors).** The spec is split into: 1. a **development-time configuration** written by the *application developer* (what the host *can* expose), and 2. a **deployment-time configuration** written by the *extension manager* (what is *actually allowed* for a deployment / entry). ------ ### 3.1 Development-time EIM (written by the application developer) Because the exposed hooks/objects are tightly coupled to the host code, the development-time EIM is created while developing the host and defines three kinds of entries: **state capabilities**, **function capabilities**, and **extension entries**. **State capabilities**: permission to **read(var)** or **write(var)** for a host variable (global state in the host). **Function capabilities**: permission to call a host function, defined by a **name**, **prototype**, and **constraints** (pre/post conditions). **Constraints**: encode argument/return relationships and semantic facts; the paper lists supported categories like allocation facts, IO facts, annotation facts (akin to Linux eBPF annotations), and read/write facts over fields of arguments. **Extension entries**: the hook points an extension can attach to; EIM entries are described as function-based interposition points (an entry names the host function and its prototype).  ------ ### 3.2 Deployment-time EIM (written by the extension manager) Deployment-time EIM chooses the actual tradeoff for a deployment by defining **Extension Classes**. Each class binds to one extension entry and lists the **allowed capabilities** for extensions in that class. Allowed capabilities can include: - capabilities from the development-time spec, - state capabilities over the entry’s arguments, - **resource capabilities** (hardware-resource limits), which EIM defines as **instructions** and **memory**. They illustrate this with the Nginx example: the deployment-time spec defines classes per entry whose allowed sets include state caps (e.g., read/write on arguments), resource caps (e.g., instruction bounds), and function caps—intended to block past extension failures like livelock/outages.  ### 4 bpftime Design