eHashPipe: Kernel Resource Observability

• eHashPipe is a kernel observability system that leverages eBPF and the HashPipe sketching algorithm to monitor CPU and memory usage with high precision.
• It implements dual in-kernel pipelines for on-CPU time and memory allocation tracking, achieving ~0.011 s temporal resolution and reducing context-switch overhead.
• The system supports both top-k and specific PID tracking modes, providing efficient real-time insights with a small runtime footprint.

Searching arXiv for the specified paper and closely related work. eHashPipe is a lightweight, real-time resource observability system that utilizes eBPF and the HashPipe sketching algorithm to monitor system-level CPU and memory usage in kernel space. It supports two tracking modes: Top-k monitoring, which identifies the most resource-demanding processes, and specific PID tracking, which provides exact behavior for selected processes. The system implements two in-kernel eBPF pipelines, one for on-CPU time and one for memory usage, and is designed to reduce latency and context-switch overhead relative to traditional userspace polling tools while keeping the runtime footprint small (Dai et al., 11 Sep 2025).

1. Problem setting and design objective

System-level resource monitoring with both precision and efficiency is a continuous challenge. Operators and runtime systems need precise, timely monitoring of CPU and memory to control performance, detect anomalies, and manage workloads. Traditional userspace polling tools such as top, pidstat, and vmstat read /proc periodically, producing coarse-grained snapshots and incurring context-switch latency. Advanced tracers such as perf and ftrace offer detail, but they require expertise and can impose high overhead (Dai et al., 11 Sep 2025).

eHashPipe is motivated by the observation that eBPF enables safe, programmable, in-kernel observability at low overhead, while existing eBPF tools often report per-event metrics without real-time prioritization or top-k summarization across processes. The system is therefore positioned to bridge a specific gap: to deliver precise, low-latency, low-overhead monitoring that can surface the top-k resource consumers quickly and track selected PIDs exactly, including short-lived bursts that are invisible to userspace polling (Dai et al., 11 Sep 2025).

A common misconception is to treat eHashPipe as merely a direct renaming of HashPipe. The original HashPipe paper introduces HashPipe as a heavy-hitter detection algorithm for programmable data planes and explicitly notes that the term “eHashPipe” does not appear in that work. eHashPipe is instead a kernel observability system that adapts HashPipe-style heavy-hitter retention to eBPF-based CPU and memory monitoring (Sivaraman et al., 2016).

2. In-kernel architecture and data flow

The architecture is organized as two in-kernel pipelines and a lightweight userspace agent. The memory pipeline records allocation metadata at function entry, captures returned pointers at function exit, and maintains a ptr2size mapping to support accurate deallocation accounting. Each memory event is transformed into an update of the form (pid, delta_size), where delta_size is positive for allocations and negative for deallocations. This update is then fed into a multi-stage HashPipe sketch in which each stage consists of fixed-size slots keyed by a per-stage hash and each slot stores a pid and a virtual-memory estimate (Dai et al., 11 Sep 2025).

The CPU pipeline uses the sched_switch tracepoint. When a thread is scheduled in, its timestamp is recorded; when that thread is scheduled out, the delta since the recorded timestamp is computed and attributed to its process as on-CPU time. These additive updates are supplied to a HashPipe pipeline structured like the memory pipeline. The design therefore turns scheduler activity into per-process on-CPU accounting without userspace polling (Dai et al., 11 Sep 2025).

The principal BPF data structures are explicitly identified. They include a temporary entry map for allocation-function entry, a ptr2size BPF hash map from pointer to size, one fixed-capacity stage map per HashPipe stage, a global BPF_HASH keyed by thread ID for sched_switch timestamps, dedicated priority PID maps for exact tracking, and a SpecifiedPIDs set storing the PIDs that should bypass sketch-based approximation. The paper reports “5 inner maps (2000 entries each, 40 bytes per entry)” for the memory module, consistent with five stages and 2000 slots per stage. It also notes that the design uses hash maps and fixed-size array maps for the stages, and does not mention per-CPU maps or ring buffers (Dai et al., 11 Sep 2025).

The userspace component periodically reads the stage maps and priority PID maps to render the top-k list and exact per-PID statistics. For the accuracy experiments, the detection frequency was once every 2 seconds to align with top’s sampling interval. For responsiveness demonstrations, eHashPipe targeted 0.01 s and achieved approximately 0.011 s effective resolution (Dai et al., 11 Sep 2025).

3. HashPipe lineage and the eHashPipe update rule

eHashPipe inherits its central summarization mechanism from HashPipe, which was originally proposed for heavy-hitter detection entirely in the data plane. HashPipe maintains a pipeline of hash tables and retains counters for heavy flows while evicting lighter flows over time, under the constraint of one read-modify-write per stage and single-pass feed-forward processing (Sivaraman et al., 2016).

In eHashPipe, the sketch is expressed as a sequence of $d$ stages, each with $N$ slots. For an event, the per-stage index is computed as

$h_i = (a_i \cdot pid + b_i) \bmod N .$

If the resident slot already contains the same pid, the counter is updated; if the slot is empty, the incoming entry is inserted; otherwise, the event triggers a stage-dependent collision policy. Stage 0 applies forced eviction, described in the paper as “always kick out,” so every new flow enters the pipeline. For stages $i > 0$, the incoming entry displaces the resident only if its value is larger; otherwise, it halts (Dai et al., 11 Sep 2025).

This update rule differs in an important way from the original networking formulation. In the memory pipeline, counters support both positive and negative updates because allocations increase memory usage and deallocations decrease it. Accurate deallocation accounting depends on looking up the pointer in ptr2size, decrementing the corresponding process value, and deleting the pointer entry to avoid staleness. In the CPU pipeline, by contrast, counters are monotonic additions of on-CPU deltas computed from sched_switch (Dai et al., 11 Sep 2025).

The heavy-hitter intuition remains the same across the two settings. The cascade ensures that heavier entries replace lighter ones deeper in the pipeline; over time, heavy hitters persist across stages and remain resident in slots, and aggregating and sorting stage slots yields an approximate top-k set that closely tracks the true heavy hitters. The paper states the update cost as constant time per stage, with worst-case $O(d)$ per event (Dai et al., 11 Sep 2025).

4. Tracking modes and probe instrumentation

eHashPipe exposes two tracking modes. In Top-k mode, the sketch maintains approximate counts in stage slots, and the userspace agent collects stage entries and sorts them by counter to produce the top-k set. In per-PID mode, events whose pid is in SpecifiedPIDs are routed before sketch insertion to priority_vm for memory or to per-PID on-CPU maps for CPU. These priority maps are not subject to eviction and track true values (Dai et al., 11 Sep 2025).

The instrumentation strategy is deliberately practical. For memory usage, eBPF probes are attached to 12 allocation functions, while deallocation hooks include free, munmap, and kernel-level memory frees. At allocation entry, the system records (TID, size) in a temporary map; at allocation exit, it captures the returned pointer and stores ptr -> size in ptr2size so that a future deallocation can subtract the correct amount. The paper states that thread IDs are used to reduce collisions in multithreaded workloads and that accounting is aggregated to PID for top-k reporting (Dai et al., 11 Sep 2025).

For on-CPU time, the system uses sched_switch, stores the last scheduled-in timestamp in a global BPF hash keyed by thread ID, and computes the delta when the same thread is scheduled out. The tracepoint provides the scheduled-out task, while the scheduled-in thread can be identified via bpf_get_current_pid_tgid(), giving both TID and PID. The resulting per-thread durations are attributed to the process (PID), and because on-CPU time is monotonic, the CPU pipeline performs additive updates only (Dai et al., 11 Sep 2025).

The implementation also addresses high-churn conditions. The paper notes that very high allocation and deallocation rates, such as those in C++ ML frameworks, introduce contention and races, and that the implementation uses atomic operations __sync_fetch_and_add/sub to mitigate consistency issues. This suggests that correctness under concurrency was treated as a first-order systems concern rather than as a purely algorithmic property (Dai et al., 11 Sep 2025).

5. Accuracy, responsiveness, and empirical footprint

The evaluation was conducted on Ubuntu 22.04 with Linux kernel 5.15.167, using a 6-core/12-thread x86_64 CPU (max 4.3 GHz) and 16 GB RAM. Workloads included synthetic stress such as fork bombs and high-frequency allocations and context switches, along with nginx and CPU-only PyTorch training of ResNet-34 on CIFAR-10. The memory module used 5 inner stage maps (2000 slots each, ~40 B per slot), and the accuracy measurements used a 2-second detection frequency (Dai et al., 11 Sep 2025).

The paper defines top-k accuracy as the percentage overlap between the set of top-k PIDs reported by eHashPipe and the top-k list from top at the same timestamp: $$\text{Accuracy}(k) = \frac{|\hat{S}_k \cap S_k|}{k},$$ where $S_k$ is the top-k set from top and $\hat{S}_k$ is eHashPipe’s top-k set (Dai et al., 11 Sep 2025).

$k$	CPU accuracy	Memory accuracy
1	100.0%	100.0%
5	100.0%	100.0%
10	100.0%	90.0%
20	95.0%	90.0%
30	93.3%	83.3%

For responsiveness, eHashPipe targeted 0.01 s and achieved approximately 0.011 s effective resolution, while top saturated at approximately 0.15 s, yielding about 14× finer temporal resolution. The paper attributes the improvement to in-kernel, event-driven updates that react near-immediately to context switches and allocation or deallocation calls, thereby surfacing short-lived bursts and cyclic behavior that userspace polling smooths out or misses (Dai et al., 11 Sep 2025).

The reported footprint and overhead are similarly explicit. The memory module uses approximately 1.2 MB, described as 5 × 2000-slot stage maps, 40 B/slot + aux maps, while the CPU module uses approximately 183 KB. CPU overhead, measured via perf stat, is reported as approximately 20% in the evaluation setup for the eBPF process (Dai et al., 11 Sep 2025).

6. Limitations, deployment constraints, and future directions

The system’s limitations follow from both sketching and eBPF execution constraints. Stage maps are fixed-capacity, so collisions and evictions can displace entries; heavy hitters should persist, but small flows can churn. The paper further notes that negative updates in the memory pipeline complicate eviction dynamics. Although the effective temporal resolution is far better than top, extremely short bursts, especially those shorter than the effective ~11 ms resolution or masked by scheduling behavior, may still be underrepresented (Dai et al., 11 Sep 2025).

Verifier constraints are also central. Limited stack, instruction count, and analyzable control flow impose strict bounds, and the pipeline loop must be fully unrolled with #pragma clang loop unroll(full). Excess stage count or large map sizes can therefore cause verifier rejection or load failure. The main tunables are the number of stages $d$, the number of slots per stage $N$0, sampling frequency, the choice of hash functions $N$1, and the decision to key intermediate state by PID or TID (Dai et al., 11 Sep 2025).

Deployment requirements are standard for an eBPF-based kernel monitor but remain operationally important. The paper identifies the need for eBPF support with kprobes and tracepoints, BPF map support for hash maps and array maps, and helpers such as bpf_get_current_pid_tgid and a timestamp helper such as bpf_ktime_get_ns. Loading eBPF programs typically requires CAP_BPF or root privileges. Portability depends on symbol availability and kernel configuration; the reported implementation was tested on Linux kernel 5.15.167 on Ubuntu 22.04 (Dai et al., 11 Sep 2025).

The paper does not provide a public repository URL or specific commands or configuration scripts. Its stated future work is to extend the framework to additional metrics relevant to distributed ML workloads, including GPU utilization, memory bandwidth, and inter-node communication latency, and to integrate these signals for holistic profiling and automated optimization such as adaptive scheduling, batch-size scaling, and communication tuning. It also proposes applying sketch-based low-overhead telemetry to user-level applications in other domains, including EHR systems (Dai et al., 11 Sep 2025).