Kops: Safely Extending the eBPF Compilation Pipeline with Native Operations

Abstract: eBPF safely extends OS kernels in domains such as networking, observability, and security. The safety comes from an in-kernel compilation pipeline where a verifier checks every program, and a kernel just-in-time compiler (JIT) translates the verified bytecode to native code. The kernel keeps the JIT simple to stay trustworthy, translating one bytecode instruction at a time in a single pass. This single-pass design misses optimization opportunities, so eBPF runs up to twice as slow as natively compiled code in our characterization. Adding optimizations to the kernel JIT directly requires upstream acceptance and a long release cycle, enlarges the trusted computing base (TCB), and grows the per-architecture kernel code. To address this, we present Kops, an extension interface that lets userspace compilers and kernel modules introduce new operations without modifying the kernel core, while keeping a minimal trusted computing base (TCB). Each operation has two forms, a proof sequence of vanilla eBPF instructions that the existing verifier checks and a native emit of machine instructions that the JIT compiles. Because the verifier checks the proof sequence, the native emit is the only per-operation addition to the TCB. Hardware idioms are the lowest-hanging fruit for this interface. With Kops, we build EInsn, seven operations such as rotate and conditional select that CPUs execute as single instructions. Lean 4 proofs show that each native emit computes the same result as its proof sequence. On x86-64 and ARM64, EInsn speeds up eBPF microbenchmarks by up to 24% and production applications by up to 12%. The same interface also supports whole-program native replacement, reaching 2.358x at the cost of a larger TCB.

• The paper introduces a dual-form operation abstraction combining a verifiable eBPF proof sequence with a native machine emit to safely optimize JIT performance.
• It uses formal proofs in Lean 4 to ensure semantic equivalence between eBPF and native instructions, preserving kernel safety.
• Microbenchmarks and case studies in Cilium and Katran show notable speedups and throughput improvements without modifying the kernel core.

Kops: Safely Extending the eBPF Compilation Pipeline with Native Operations

Introduction and Motivation

The extended Berkeley Packet Filter (eBPF) has become foundational for kernel extensibility in contemporary operating systems, supporting domains spanning networking, observability, and security. eBPF’s safety model relies on an in-kernel bytecode verifier, followed by a simple single-pass per-opcode just-in-time (JIT) compilation pipeline. This architectural simplicity promotes trustworthiness by keeping the trusted computing base (TCB) small, but it leads to significant missed optimization opportunities. The kernel JIT emits suboptimal code for instruction sequences, especially for complex hardware idioms not directly mapped by the eBPF ISA, resulting in execution slowdowns up to 2x relative to native code.

While enhancing kernel JITs with more aggressive optimizations is possible, it is not practical in production due to constraints on kernel modifications, verification requirements, and portability concerns. Recent efforts in userspace optimizers perform advanced transformations, but they remain restricted to the bounds of the eBPF instruction set and cannot synthesize single-instruction hardware idioms at the native level. Thus, there exists a systematic gap between the eBPF value proposition of safety and its performance cost.

The Kops Mechanism: Architecture and Design

Kops addresses the eBPF-native performance gap by introducing a principled extension interface that cleanly separates optimization capabilities from core kernel modifications. The key innovation is the dual-form operation abstraction: each extension supplies both a “proof sequence” (a sequence of vanilla eBPF instructions checked by the incumbent verifier) and a “native emit” (an optimized machine instruction expansion executed by the JIT). During compilation, a userspace recognizer identifies patterns in the bytecode suitable for replacement and rewrites them as calls to these extensible operations.

The eBPF verifier is left unchanged, analyzing the lowered proof sequence for safety properties. At load time, as long as the proof sequence passes the standard safety checks and adheres to structural constraints (single-entry, single-exit, no nested extended instructions), the kernel can safely reconstitute the call to the extension, delegating the final code emission to the supplied native emit. Importantly, a formal proof (in Lean 4, for this work) is required to establish semantic equivalence between each operation’s proof sequence and its native emit, thereby maintaining the verification guarantees.

Figure 1: The eBPF execution pipeline, illustrating user-space compilation, kernel verification, and JIT translation.

A crucial aspect is that only the native emit becomes an increment to the TCB. The recognizer and proof sequence sit outside the core trust boundary, so even incorrect recognizer patterns cannot lead to unsafe kernel behavior. This enables rapid innovation and deployment of new operations as kernel modules, minimizing disruption to the kernel core and per-architecture JIT implementations.

Figure 2: Comparison between the original eBPF pipeline and the Kops pipeline, showing where Kops introduces extensibility and optimized native code emission.

EInsn: Hardware-Idiom Extension Set

To concretely demonstrate Kops, the authors introduce EInsn, an extensible set of seven hardware-idiom operations. These cover critical primitives such as rotate (ROL/ROR), conditional select, bit-extract, byte swap, prefetch, bulk copy, and address computation (LEA). The rationale is that these idioms are frequently encountered in real workloads but suffer from ISA-induced inefficiencies under standard eBPF JIT translation. Each EInsn operation comes with a formally verified proof sequence and a native emit for x86-64 and ARM64, with architecture-specific adaptations as needed.

Figure 3: Illustration of code-size bloat: an idiomatic 64-bit rotate compiles to eight eBPF instructions and inflates to 15 native instructions in the kernel JIT; native compilation produces a single ROL.

Lean 4 formalization ensures that, for each hardware idiom, the observable architectural state post-execution is identical whether running the proof sequence or the native emit. For instance, rotate, select, and extract equivalence check only register values, while memory operations such as byte swap and bulk copy additionally constrain the memory projection.

Evaluation and Results

Microbenchmark Suite

The design is evaluated on 27 pure-bytecode microbenchmarks drawn from real-world networking, tracing, and security workloads, encompassing diverse idiomatic computation kernels. Kops/EInsn delivers geometric mean speedups of 1.242x on x86-64 and 1.222x on ARM64 over the stock kernel eBPF JIT. Notably, these optimizations recover up to 42% of the measured eBPF-native performance gap on x86-64 and 18–19% on ARM64, confirming that a bounded idiom set covers the dominant sources of kernel JIT losses.

Figure 4: Per-case speedup over the kernel eBPF JIT for the 27 microbenchmarks, on both x86-64 and ARM64.

Figure 5: x86-64 KVM EInsn-enabled microbenchmark execution, reporting distribution of speedups and correlation with code-size reductions.

Real-Application Case Studies

EInsn was deployed in production-grade workloads: Cilium (complex Kubernetes datapath) and Katran (high-rate XDP load balancer). Measured end-to-end improvements reached 7.4% for Cilium and 7.3% for Katran compared to the stock pipeline. Notably, these gains were achieved without any kernel-core code changes or instruction-set extensions, demonstrating the viability of Kops/EInsn in realistic operational environments.

Figure 6: Application-level EInsn policy probes, showing throughput ratios versus baseline for Cilium and Katran, and the impact of targeted versus maximal site enabling.

The results also revealed that “maximal coverage” (enabling all matched extension sites) does not always translate into the maximal throughput; optimal policies require careful descriptor-family selection to ensure net benefit on per-workload bases.

Trusted Computing Base and Performance Bound

By leveraging the Kops interface for whole-program native replacement (i.e., using the verifier only to check the eBPF proof sequence, trusting the provided native emit), the authors demonstrate an upper bound: Cilium throughput improved by 2.36x over eBPF JIT. This level of speedup does not preserve the original TCB and demands whole-program trust, in contrast to EInsn’s bounded exposure.

Implications, Limitations, and Future Directions

Kops generalizes the notion of proof-carrying code to the kernel extension domain, but with a strong emphasis on minimal TCB augmentation and compositional extensibility. By architecting the interface around dual-form operations with rigorous, offline, per-operation formal proofs, Kops circumvents the substantial risks and maintenance burdens involved in directly inlining optimizer logic or instruction-set changes in kernel space.

Practically, Kops enables incremental deployment of hardware-specialized operations targeting emergent microarchitectures, without stagnating on the kernel release cycle or drifting away from the core verifier’s mature analysis. The policy experiments also highlight the necessity for workload-adaptive selector logic, perhaps coupled with lightweight in-kernel or JIT-side profitability modeling, to avoid negative performance transfers from unhelpful replacements.

Theoretically, the mechanism could enable a broader class of kernel optimizations—spanning complex program regions, software idioms, or even user-defined fast-paths—without sacrificing the compositional safety or analyzability afforded by eBPF’s design.

Conclusion

Kops establishes an extensible, formally justified, and minimally invasive approach to extending eBPF’s compilation pipeline with native operations. Instantiated via EInsn, this approach yields significant speedups on microbenchmarks and meaningful gains on production packet-processing workloads. The construction retains core kernel safety guarantees, makes only bounded additions to the TCB, and empowers the broader eBPF ecosystem to deploy optimization or specialization logic outside the glacial pace of kernel instruction-set revision. As the line between hardware, operating system, and JIT-accelerated kernel platforms continues to blur, Kops provides a technically robust template for sustainable, safe, architecture-aware kernel extensibility.