OffloadFS: Hybrid Computation & Storage Offloading
- OffloadFS is a dual-use concept: in 2025 it refers to middleware orchestrating semantic offloading between foundation models and executable specifications, while in 2026 it denotes a lightweight file system for NVMeoF-based disaggregated storage.
- It offloads tasks based on capability fit, assigning unstructured, probabilistic tasks to foundation models and precise, verifiable tasks to executable specifications to optimize system performance.
- Its storage implementation leverages idle compute on storage nodes for near-data processing, achieving performance improvements up to 3.36× over traditional systems.
OffloadFS is a term used in recent arXiv literature for two distinct but related proposals in computation offloading. In the 2025 perspective paper "An Infrastructure Software Perspective Toward Computation Offloading between Executable Specifications and Foundation Models" (Ran et al., 6 Jan 2025), OffloadFS denotes a middleware-oriented view of hybrid software systems in which subtasks are assigned to either foundation models (FMs) or executable specifications according to semantic fit. In the 2026 systems paper "OffloadFS: Leveraging Disaggregated Storage for Computation Offloading" (Moon et al., 15 Apr 2026), it denotes a lightweight, user-level file system for NVMe over Fabrics (NVMeoF)-based disaggregated storage that offloads I/O-intensive work to a storage node for near-data processing, with the option to offload to peer compute nodes as well. The shared theme is selective delegation of computation to the substrate best matched to the task, but the two uses differ sharply in abstraction level, scope, and implementation status.
1. Terminological scope and dual usage
The term “OffloadFS” spans both a conceptual infrastructure-software framework and a concrete storage-system implementation (Ran et al., 6 Jan 2025, Moon et al., 15 Apr 2026).
| Aspect | 2025 usage | 2026 usage |
|---|---|---|
| Primary setting | computation offloading between executable specifications and foundation models | NVMeoF-based disaggregated storage |
| Offloading basis | semantic nature, reliability requirements, and resource constraints | near-data execution of I/O-intensive tasks |
| Form | perspective piece | lightweight, user-level file system |
The 2025 usage treats offloading as assignment between two computational paradigms: FMs on one side, executable specifications or symbolic programs on the other. Its novelty is explicitly described as not ordinary offloading based on hardware or latency alone, but “semantic offloading,” where the assignment is driven by task semantics, precision needs, uncertainty tolerance, verification needs, resource availability, and latency/cost tradeoffs. By contrast, the 2026 usage addresses a specific systems problem in disaggregated storage: storage targets in NVMeoF often retain idle CPU and memory that conventional protocols do not exploit, while applications such as RocksDB and machine-learning pipelines have expensive background I/O tasks that are attractive candidates for near-data execution.
This terminological overlap can cause confusion. One common misconception is to treat the two proposals as successive versions of a single system. The available descriptions do not support that interpretation. The 2025 paper is a perspective on hybrid FM-native software development, whereas the 2026 paper is a concrete design for a single application or single initiator-centric access model over shared storage.
2. OffloadFS as semantic offloading between executable specifications and foundation models
In the 2025 perspective, OffloadFS is motivated by the claim that foundation models and executable specifications are complementary rather than substitutive (Ran et al., 6 Jan 2025). FMs are characterized as strong at pattern recognition, handling unstructured data, ambiguous, uncertain, or creative tasks, and natural-language interaction. Executable specifications or symbolic programs are characterized as strong at exact computation, deterministic control, formal reasoning, constraint satisfaction, and verification. The objective is to build reliable FM-based software systems by strategically distributing computation between these paradigms.
Offloading in this setting is defined as assigning subtasks of a software system to either a foundation model or an executable specification based on the subtask’s semantic nature, reliability requirements, and resource constraints. This extends the classic cloud/offloading idea. Traditional offloading moves work between devices or the cloud primarily for performance or resource reasons; OffloadFS moves work between FMs and symbolic programs based on capability fit.
The paper organizes the comparison in terms of asymmetric strengths and weaknesses. FMs handle unstructured inputs, recognize patterns at scale, generalize across messy or ambiguous inputs, produce creative and fluent outputs, and are easy to use for natural-language-driven tasks. Their weaknesses include imprecision, unreliability on strict logical reasoning, difficulty of formal verification, possible constraint violations or hallucination, and the expense of training and maintenance. Executable specifications are precise, deterministic, formally verifiable, and effective for structured logic and exact computations, while remaining easier to reason about for correctness and relatively stable and maintainable via code changes. Their limitations are poor handling of ambiguity and unstructured data, a steep learning curve, lower flexibility than FMs, and greater difficulty adapting to informal or underspecified requirements.
The proposed division of labor is therefore explicit: the FM handles what is fuzzy, statistical, or perceptual, and symbolic programs handle what is exact, formal, or safety-critical. The illustrative examples are correspondingly hybrid. In image processing, an FM identifies complex patterns while symbolic code performs precise geometric transforms. In natural-language processing, an FM generates coherent text while symbolic code performs rigorous grammar checking. A broader example is a system that uses an FM to interpret an ambiguous user request, then uses symbolic logic to enforce business rules and symbolic code to validate or correct FM outputs.
3. Middleware mechanisms, task decomposition, and open technical problems
The 2025 paper proposes an infrastructure software layer that acts as middleware for orchestrating computation offloading in FM-native systems (Ran et al., 6 Jan 2025). The framework contains eight key mechanisms: context analysis and task decomposition; resource assessment and allocation; intelligent offloading decisions; data flow management; execution coordination; quality assurance and verification; adaptive optimization; and developer interfaces.
The decomposition logic is conceptual rather than algorithmic. Complex tasks are split into subtasks according to their computational nature, including perception or pattern tasks, reasoning or logic tasks, constraint-solving tasks, generation tasks, and verification tasks. Assignment then depends on task type, constraint strictness, need for formal correctness, data modality, and hybrid composition. Unstructured or uncertain tasks tend toward FMs; exact, deterministic, or verifiable tasks tend toward symbolic code. Strict constraints favor symbolic execution, while flexible constraints favor FMs. When formal correctness is required, the task is assigned to symbolic code or FM outputs are validated by symbolic checks. Natural language, images, and noisy data favor FMs; structured data, rules, code, and formulas favor symbolic programs.
The paper does not present a formal algorithm, explicit cost function, mathematical program for resource allocation, symbolic notation for offloading decisions, or formal control-loop equation. Instead, it describes a qualitative optimization loop: observe task and system state, choose an execution split, run tasks, validate and measure outcomes, and update strategy for future tasks. Adaptive optimization is expected to learn from previous runs, adjust offloading decisions based on observed performance, adapt to resource availability changes, and improve efficiency over time. Likely optimization aids mentioned include caching, prediction mechanisms, reducing unnecessary data transfer, and minimizing context-switch overhead.
Six major challenge areas delimit the research agenda. Semantic-gap resolution concerns the mismatch between probabilistic, distributed FM representations and precise, deterministic symbolic specifications, including representation mismatch, context preservation, and semantic consistency across abstraction levels. Resource management and optimization concern dynamic allocation between FM inference and symbolic execution, memory heterogeneity, and energy efficiency. Reliability and verification concern verifying outputs that combine probabilistic and deterministic components, controlling error propagation, and runtime verification of FM results. Performance and scalability concern reducing offloading and context-switch overhead, scaling to large systems and data volumes, and using caching or prediction to avoid waste. Security and privacy concern sensitive data crossing system boundaries, larger attack surfaces, and privacy-preserving processing through FMs. Development and deployment concern debugging and testing for hybrid probabilistic-deterministic systems, version control for both FMs and specifications, and deployment tooling. This suggests a research program centered less on a single placement policy than on the runtime substrate needed for hybrid execution.
4. OffloadFS as a user-level file system for disaggregated storage
The 2026 paper defines OffloadFS as a lightweight, user-level file system for a specific setting: a single application or a single initiator node accessing NVMeoF-based disaggregated storage, where the storage node has spare CPU and memory that conventional storage protocols do not exploit (Moon et al., 15 Apr 2026). Its central idea is to convert that underused compute into a near-data execution substrate for I/O-intensive work such as RocksDB compaction, MemTable flushes, or image preprocessing, while avoiding the heavy machinery of shared-disk file systems.
The architecture is initiator-centric. On the initiator node, the inode table and extent management remain solely on the initiator. An Extent Manager allocates extents, tracks free space, manages per-file extent trees, and handles defragmentation. A Task Offloader decides when to offload I/O work and sends RPCs to a remote node. The application or OffloadFS code provides gRPC stubs and skeletons for the offloaded logic. On the target node, an Offload Engine receives offloaded tasks via gRPC and executes them using two APIs, offload_read() and offload_write(). These APIs use SPDK to perform direct user-level I/O to the blocks supplied by the initiator. The target may cache blocks in an Offload Cache to avoid rereading hot data.
The execution model sharply separates planning from work. The initiator allocates inode, extent, and block addresses, determines which blocks the remote task may access, and sends block addresses and task metadata to the remote node. The target reads the authorized blocks, computes on them, writes output blocks if needed, and returns results or metadata to the initiator. Offloading can therefore cover reads and computation on existing extents, updates to existing extents, and returning task results, without letting the target manipulate global filesystem metadata.
OffloadFS is not limited to the storage node. Because the same NVMeoF volume can be mounted by multiple initiators, the application can also offload tasks to a peer compute node that has access to the same volume. The paper is explicit that this is not a fully decentralized shared-disk model. The application chooses where to place the work, and OffloadFS provides the block-level access and RPC framework.
5. Consistency model, cache behavior, and load control
A defining design choice in the 2026 system is the avoidance of distributed lock management (Moon et al., 15 Apr 2026). OffloadFS does not require distributed inode locking, directory locks, block-level cross-node lock negotiation, or cache-coherence message exchange between daemons because conflicting concurrent access is prevented by construction. Only the initiator manages metadata; offloaded tasks are given only explicit block addresses that the initiator has already allocated and authorized; the initiator tracks which blocks are in use by an offloaded task; and the initiator does not access those blocks again until the task completes. The result is much lighter than OCFS2 or GFS2 in the targeted setting.
Cache management is similarly simplified. The initiator does not use cache within OffloadFS itself, although applications may add their own caching. The Offload Cache lives on the target node and is used for offloaded reads. Blocks may become stale, but instead of using fine-grained invalidation through distributed messages, OffloadFS assigns coherence responsibility to the application or initiator. The initiator can tell the target to bypass cache, or it can pass file modified time and let the target compare against cached data age. The coherence regime is therefore intentionally coarse-grained.
This cache separation is also used to reduce interference between different I/O threads. The paper identifies cache pollution as a major issue in LSM trees because foreground reads and background compaction share the same page cache in conventional setups, while compaction reads victim SSTables sequentially and writes new SSTables that are soon deleted. OffloadFS separates roles: foreground I/O stays on the initiator, background I/O can be offloaded to the target, the target uses its own Offload Cache, and OffloadDB can additionally use an L0 cache on the initiator to keep short-lived MemTables and L0 SSTables local. The intended effect is that foreground query data stays near the application while compaction data can live on the target and be discarded with less harm.
Placement and admission control are handled by two informal mechanisms rather than a formal scheduler. In threshold-based offloading control, if target CPU usage exceeds a threshold, new offload requests are rejected and the initiator executes locally instead. In token-based offloading control, the storage node circulates a fixed number of tokens, only nodes holding a token may send offload requests, and tokens expire after a time-to-expire interval. The paper reports that both mechanisms prevent overload once storage-node CPU becomes saturated, while token-based control generally performs slightly better and more fairly, albeit with timeout tuning.
6. Application specializations, evaluation, and limitations
The 2026 system is instantiated in two application-level designs: OffloadDB and OffloadPrep (Moon et al., 15 Apr 2026). OffloadDB is a RocksDB variant built on OffloadFS. RocksDB has four major I/O activities—WAL logging for writes, flushing MemTables to L0 SSTables, compaction or merge-sorting of SSTables, and MANIFEST updates—but OffloadDB offloads only MemTable flush and compaction, keeping WAL management and MANIFEST management on the initiator because those are frequent foreground-sensitive operations and the initiator should remain responsible for metadata commits. A key optimization is Log Recycling: instead of sending MemTable contents twice, once to WAL and again during flush, OffloadDB sends compact metadata and offsets and lets the target reconstruct L0 SSTables by reading the WAL. The L0 cache keeps immutable MemTables and L0 SSTables locally until they are compacted away, with the stated purposes of avoiding storage of short-lived L0 data on the remote target, helping foreground reads, and reducing useless round-trips to disaggregated storage.
OffloadPrep extends the same substrate to machine-learning data preprocessing. It offloads image preprocessing tasks such as resize, crop, flip, and rotate to the storage node, a peer compute node, or both. Unlike tf.data.service, Cachew, Sophon, Pecan, or FusionFlow, it does not try to be a scheduler for distributed preprocessing workers; it uses OffloadFS as the execution substrate and leaves placement and load control to OffloadFS mechanisms.
The evaluation uses a 9-node cluster with 8 compute nodes and 1 NVMeoF storage node, a Mellanox FDR 40/56GbE InfiniBand switch, compute nodes with 2 Intel Xeon Gold 5115 processors, 20 vCPUs, and 64 GB DRAM, and a storage node with 2 Intel Xeon Silver 4215 processors, 16 vCPUs, 128 GB DRAM, and 24 Samsung PM9A3 U.2 NVMe SSDs; the storage software is PoseidonOS v0.12.0. For OffloadDB, workloads include 32 client threads, 20 background threads, 64 MB SSTables, a 200 GB database, 24-byte keys, 1 KB values, 20 million requests in the query phase, and YCSB workloads including write-heavy and mixed workloads. For OffloadPrep, the setup uses 4 preprocessing threads per initiator, an OpenImage 10 GB dataset, a minibatch of 64 images, and caching disabled for preprocessing because the dataset is larger than memory. Baselines include OCFS2, GFS2, RocksDB, SpanDB, Hailstorm, and variants with direct I/O or offload cache enabled or disabled.
The headline results are that OffloadDB improves performance by up to over OCFS2 and OffloadPrep improves performance by up to over OCFS2. OffloadFS is reported to be about faster than OCFS2 even when not offloading compaction, because it is a lightweight user-level SPDK file system. With all compactions offloaded, OffloadFS reaches 379K ops/sec compared to OCFS2’s best 110K ops/sec. Log Recycling improves write throughput by up to over offloading without it. The L0 cache reduces SSTable accesses by and increases throughput by in one configuration. For OffloadPrep, offloading image preprocessing only to the target node yields 15.19 s versus 28.18 s for OCFS2. The paper also reports that OffloadPrep benefits more from peer compute nodes than from the storage node for pure preprocessing because preprocessing is more CPU-intensive and the compute node is stronger than the storage node, while using both storage and peer nodes is better than using peer only.
The limitations are explicit. OffloadFS is not a general shared-disk file system. It targets a single application or single initiator-centric access model; the initiator is responsible for metadata and deciding which blocks are safe to offload; offloaded tasks are block-address based and constrained to authorized extents; and the workload must be amenable to separating foreground and background I/O. It does not support directory operations or metadata-heavy remote actions, and it cannot offload truncate, fallocate, stat, or ioctl on the target. Cache coherence is coarse-grained and application-managed. Scan-intensive workloads are not optimized well; scan-heavy YCSB workload E is the stated case where OffloadDB underperforms vanilla RocksDB. The suggested future directions include improving sequential scan performance, making load-control or admission policies smarter, extending offload to other underused resources such as computational SSDs or DPUs, and integrating with more applications. In relation to the 2025 perspective, a plausible implication is that OffloadFS has become a useful label for a broader systems question: how to construct an orchestration layer that assigns computation to the substrate with the most appropriate tradeoff among precision, locality, synchronization cost, and available resources.
