---
title: 'Kops: Extensible eBPF JIT Optimization'
url: https://www.emergentmind.com/topics/kops
type: topic
---

# Kops: Extensible eBPF JIT Optimization

Kops is an extension interface for the eBPF compilation pipeline that allows userspace compilers and kernel modules to introduce new operations without modifying the kernel core, while keeping a minimal trusted computing base. It is designed for the standard eBPF setting in which a verifier checks every program and a kernel just-in-time compiler translates verified bytecode to native code. The motivating observation is that the kernel JIT intentionally translates one bytecode instruction at a time in a single pass to remain trustworthy, but this simplicity misses optimization opportunities; in the paper’s characterization, eBPF runs up to twice as slow as natively compiled code. Kops addresses that tension by giving each extension operation two forms: a proof sequence of vanilla eBPF instructions that the existing verifier checks, and a native emit of machine instructions that the JIT compiles [2606.24213].

## 1. Problem formulation and design target

Kops is motivated by a specific structural limitation of contemporary eBPF. The existing safety model is built around a fixed eBPF ISA, an in-kernel verifier with opcode-by-opcode transfer functions, and a simple JIT that performs per-opcode translation in a single pass. This arrangement concentrates safety reasoning in the verifier and keeps the JIT small, but it also prevents recovery of multi-instruction idioms and architecture-specific fast paths once source-level constructs have been lowered into ordinary eBPF bytecode [2606.24213].

The performance cost is quantified in the paper’s characterization. Across 27 pure-bytecode microbenchmarks, the eBPF pipeline is **1.57× slower than native on x86-64** and **1.98× slower than native on ARM64**. The paper further reports that in-kernel execution overhead itself is small—about **2% on x86-64** and **5% on ARM64**—and that recompiling the same verified bytecode with an optimizing compiler in userspace recovers **93–94%** of the eBPF-native gap. This places the bottleneck squarely in the JIT pipeline rather than in the abstract eBPF execution model [2606.24213].

A canonical example is a 64-bit rotate with variable shift. Direct native compilation emits a single instruction—`ROL` on x86-64 or `ROR` on ARM64—whereas the eBPF path expands the same operation into **8 eBPF bytecode instructions**, and the stock x86-64 kernel JIT emits **15 machine instructions** for that sequence. Kops is therefore not a general-purpose compiler replacement; it is a mechanism for recovering bounded, profitable operations that the fixed-ISA verifier and single-pass JIT cannot otherwise express efficiently [2606.24213].

## 2. Two-form operation model

The central abstraction in Kops is that every extension operation has **two forms**. The first is a **proof sequence**, which is a fragment of ordinary vanilla eBPF instructions. The second is a **native emit**, which is the machine instruction sequence the JIT should emit instead of compiling the proof sequence literally [2606.24213].

This design preserves the existing kernel safety story. The verifier never reasons about new operation semantics directly. Instead, it verifies only the proof sequence, expressed entirely within the existing eBPF ISA. The native emit is the optimized code that actually runs after JIT compilation. Because the verifier checks the proof sequence, the paper’s key TCB claim follows directly: **the native emit is the only per-operation addition to the trusted computing base** [2606.24213].

The proof sequences are deliberately local and ISA-conservative. The paper gives representative examples: rotate as `two shifts + or`, conditional select as `branch + move`, bit extract as `shift + and`, byte swap as `load + byte swap`, prefetch as `no-op (JA +0)`, bulk copy as `loads + stores`, and LEA as `shift + adds`. This makes Kops resemble a proof-carrying mechanism in effect, but without introducing a new proof checker into the kernel [2606.24213].

A common misconception is that Kops adds new verifier-visible opcodes to eBPF. It does not. A plausible implication is that its compatibility with the existing verifier depends precisely on *not* expanding the verifier’s semantic domain.

## 3. Encoding and end-to-end pipeline integration

Kops does not introduce a new eBPF opcode. An extended instruction is encoded as a pair of ordinary eBPF instructions: a **call** and a **sidecar**, marked using a reserved `src_reg` value. The call carries the **32-bit BTF ID of the operation’s stub kfunc** in `imm` and a **16-bit index into `fd_array` for the module’s BTF file descriptor** in `off`. The sidecar carries a packed **52-bit payload** consisting of `dst_reg [3:0]`, `off [19:4]`, and `imm [51:20]` [2606.24213].

The end-to-end workflow begins in userspace. Before load, a recognizer rewrites supported idioms into these extended instructions. The recognizer can be implemented either as an **LLVM backend selector** during instruction selection in the LLVM eBPF backend or as a **standalone bytecode recovery pass** that re-matches idioms on already-generated eBPF bytecode. The recognizer is explicitly outside the safety boundary: if it rewrites incorrectly, the lowered proof sequence is still re-verified by the kernel [2606.24213].

Inside the kernel, Kops adds three generic stages around the existing verifier. In **lowering**, the kernel traverses the program, replaces each extended instruction with its descriptor’s proof sequence, and records the original extended instruction pair so that it can later be restored. In **verification**, the ordinary verifier runs unchanged on the lowered vanilla-eBPF program. In **restore**, the kernel restores the extended instructions after successful verification so that the JIT can emit the corresponding native code [2606.24213].

This pipeline architecture is significant because it localizes extensibility to descriptors and emitters rather than to the kernel verifier’s abstract semantics. A second common misconception is that Kops teaches the verifier what operations such as rotate or bit extract mean. It does not; the verifier sees only their eBPF expansions.

## 4. Safety invariants and trusted computing base

Kops’s safety argument is compositional. First, verifier safety remains unchanged because the verifier still checks only ordinary eBPF. Second, the structural correctness of lowering and restoration is enforced by proof-region constraints. Third, semantic equivalence between the proof sequence and native emit ensures that the JITted code preserves the eBPF-visible behavior of the verified lowered program [2606.24213].

To make restoration sound, each lowered proof region must behave like a single instruction. The paper imposes these constraints:

- **single-entry**
- **single-exit**
- **no nested extended instructions**
- **no calls inside a region**
- **no exits inside a region**
- **no backward jumps**
- **no jump may enter or leave a region except at its boundary**

The kernel validates these well-formedness conditions at load time. This prevents control-flow distortions that would otherwise invalidate the verifier’s reasoning after the proof sequence is collapsed back into an extended instruction [2606.24213].

The TCB consequences are one of Kops’s main systems results. The trusted enforcement path consists of the existing verifier, the existing JIT, and a small generic Kops patch that validates proof regions, performs lowering and restore, and dispatches native emits. The core kernel change is reported as **929 LOC** total: **487** for verifier lowering/restore/validation, **139** for BTF and registration, **121** for ARM64 JIT dispatch, **97** for x86-64 JIT dispatch, and **85** for headers [2606.24213].

The paper also formalizes the semantic proof obligation. Each ISA is given a **small-step register-transfer semantics**, and programs denote the **fold of per-instruction transitions**. Cross-ISA equivalence is factored through an **intermediate hardware-independent specification**. For each supported EInsn operation, the authors prove in Lean 4 that the eBPF expansion and the native sequence produce the same **eBPF-observable architectural state**. For register-only operations, this reduces to equality on the destination register; for memory-touching operations, it requires equality on the projected architectural state over live registers and accessed memory; for prefetch, the instruction is modeled as an architectural no-op, with microarchitectural effects outside the semantics [2606.24213].

This suggests that Kops is strongest when the extension surface is local and semantics-preserving at the architectural level. The paper is explicit that microarchitectural effects are not part of the formal model.

## 5. EInsn: hardware idioms as a case study

The paper instantiates Kops as **EInsn**, a family of seven operations chosen as “the lowest-hanging fruit” because CPUs already support them efficiently, the eBPF ISA lacks them directly, and the stock JIT emits their decomposed forms poorly. These operations are local, bounded, and architecture-specific in exactly the way Kops is meant to support [2606.24213].

| Operation | Proof sequence | Native emit |
|---|---|---|
| Rotate | `two shifts + or` | x86-64: `ROL`; ARM64: `ROR` |
| Conditional select | `branch + move` | x86-64: `CMP + CMOV`; ARM64: `TST + CSEL` |
| Bit extract | `shift + and` | x86-64: `BEXTR`; ARM64: `UBFX` |
| Byte swap | `load + byte swap` | x86-64: `MOVBE`; ARM64: `LDR + REV` |
| Prefetch | `no-op (JA +0)` | x86-64: `PREFETCHT0`; ARM64: `PRFM` |
| Bulk copy | `loads + stores` | x86-64: `MOV`; ARM64: `LDP + STP` |
| LEA | `shift + adds` | x86-64: `LEA`; ARM64: `--` |

The paper notes several architecture-specific asymmetries. Conditional select is represented as one extended instruction on x86-64 but as two on ARM64 (`TST` then `CSEL`). Byte swap similarly splits on ARM64 into `LDR` and `REV`. LEA is available only on x86-64 [2606.24213].

EInsn is important not because it solves all JIT inefficiency, but because it demonstrates that Kops can capture a meaningful class of missed optimizations without embedding an optimizing compiler into the kernel. A plausible implication is that hardware idioms provide a clean validation domain for the framework: their proof sequences are short, their native emits are compact, and their semantics are sufficiently local for formal equivalence proofs.

## 6. Implementation boundaries and portability

Kops is implemented across three domains. The **kernel core** receives the one-time **929 LOC** patch. **Userspace** contains two recognizer paths: an LLVM selector with **2,126 LOC** added in LLVM backend commits and a standalone bytecode-recovery tool with **6,967 LOC**. **Operation modules** are packaged as loadable kernel modules and provide proof-sequence routines, native emits, and BTF registration; the total module tree is reported as **14 modules on x86-64**, **11 on ARM64**, and **9,765 LOC** including helpers, tests, and operations beyond the seven EInsn operations [2606.24213].

The JIT modifications are deliberately small. The paper reports **97 lines** changed on x86-64 and **121 lines** on ARM64. Rather than making the JIT globally smarter, Kops adds a dispatch case for extended instructions; when such an instruction is encountered, the JIT decodes the sidecar payload, looks up the descriptor, and invokes the descriptor’s native emit for the current architecture. If no native emit exists for that architecture, Kops falls back to the proof sequence by rewriting the site back during load-time fixups so that the stock JIT compiles ordinary eBPF instead [2606.24213].

Portability is handled asymmetrically but cleanly. Verification remains architecture-independent because it always operates over vanilla eBPF proof sequences, while acceleration is architecture-specific through native emits. The interpreter does not support extended instructions, and kernels without a JIT reject programs containing them [2606.24213].

This separation is central to the framework’s modularity claim. Operation semantics are anchored in eBPF proof sequences, while performance is delegated to optional architecture-specific emits. That is a narrower portability target than source-level compiler portability, but it preserves the verifier-centric kernel trust model.

## 7. Evaluation, policy tradeoffs, and scope

The evaluation covers **27 pure-bytecode microbenchmarks** and two production packet-processing applications, **Cilium** and **Katran**. On microbenchmarks, EInsn reduces native code size to **0.772×** of stock JIT size on x86-64, a **22.8% reduction**, and to **0.879×** on ARM64, a **12.1% reduction**. The geometric-mean speedups over the stock kernel JIT are **1.242×** on x86-64 and **1.222×** on ARM64, corresponding to execution-time reductions of **19.47%** and **18.17%**, respectively. The paper summarizes peak gains as **up to 24% on x86-64** and **up to 22% on ARM64**, with **zero correctness mismatches** on the microbenchmarks [2606.24213].

The application-level results are smaller but still measurable. On **Cilium** under a full EInsn policy on x86-64, throughput improves by **1.074×** over stock eBPF, with **4086 EInsn sites** applied, including **2346 LEA**, **385 conditional select**, **766 endian fusion**, **2 extract**, and **587 bulk memory** sites. On **Katran** under a conservative EInsn policy on ARM64, throughput improves by **1.073×** with **21 sites** applied. The paper rounds the production class of results to **up to 12%** improvement [2606.24213].

A critical result is that **more sites is not always better**. For Cilium, disabling only `bulk_memory` reduces coverage from **4086** to **3512** sites but improves throughput from **1.074×** to **1.114×**. For Katran, a conservative selected-family policy with **21 sites** yields **1.073×** throughput, whereas a coverage-max policy with **62 sites** yields **0.995×**. The paper therefore argues for **profitability-based family selection**, descriptor-level gating, or a lightweight cost model rather than blanket extension application [2606.24213].

Kops also supports a more aggressive mode: **whole-program native replacement**. In this experiment, each eBPF program is associated with proof sequence = normal eBPF bytecode and native emit = whole-program native code compiled directly from the same C source with `LLVM -O2`. On Cilium, this reaches **2.358×** throughput over stock eBPF, and retained BPF cost drops from **488.7 ns/run** to **262.3 ns/run**, a **1.86× BPF-counter speedup**. But this comes with a much larger trust surface, including **113 native replacements**, **22 manifest no-match pass-throughs**, and **89 Cilium manifest objects across 8 native files** [2606.24213].

This establishes a spectrum rather than a single design point. At one end is stock eBPF, with the smallest trust boundary and weakest performance. In the middle is **Kops + EInsn**, which adds bounded instruction-surface extensions and makes only per-operation native emits part of the TCB. At the other end is whole-program native replacement, which offers the highest performance ceiling but trusts much more native code. The paper quantifies that EInsn recovers only **5.4%** of the upper-bound gap on Cilium, reflecting that real datapaths spend substantial time in helpers, maps, and tail calls that EInsn does not optimize [2606.24213].

Kops therefore does not eliminate the broader limitations of the eBPF JIT. The paper explicitly notes that it does not solve register allocation, instruction scheduling, or broader cross-instruction optimization. Its contribution is narrower and more structural: it introduces a verifier-compatible, modular extension point for native operations, enabling meaningful speedups while bounding trust expansion.

Source: https://www.emergentmind.com/topics/kops