DaxFS: CXL Shared Memory Filesystem

• DaxFS is a Linux filesystem for CXL-based disaggregated memory that enables direct, byte-addressable access and lock-free multi-host coordination.
• It employs hardware-supported 64-bit cmpxchg operations and a CAS-based overlay hash table to manage metadata and data consistency across hosts.
• Performance evaluations indicate that DaxFS outperforms traditional filesystems like tmpfs and ext4-dax in high-throughput sequential and random I/O benchmarks.

Searching arXiv for the primary DaxFS paper and closely related DAX filesystem context papers. DaxFS is a Linux filesystem for CXL-based, byte-addressable disaggregated memory whose defining property is that all cross-host coordination is expressed through coherent 64-bit cmpxchg operations on shared memory. It is designed for CXL 3.0 Global Fabric Attached Memory (GFAM), where multiple hosts can issue load/store operations and fabric-wide atomics to the same physical memory region, and it uses that capability to provide lock-free multi-host writes, a cooperative shared page cache, and zero-copy pointer-based access without a centralized coordinator or journaling in the hot path (Wang et al., 2 Apr 2026).

1. Context, design target, and relation to DAX

DaxFS is situated at the intersection of DAX-style direct access and CXL-coherent disaggregated memory. CXL 3.0 introduces GFAM, in which memory devices on a CXL switch fabric can be accessed by multiple independent hosts as byte-addressable memory, with hardware cache coherence maintained by a device coherence engine and with 64-bit cmpxchg atomicity preserved across host boundaries. DaxFS targets precisely that environment: a shared-memory substrate in which filesystem data and metadata can be manipulated via ordinary loads, stores, and atomics rather than RPCs or block I/O (Wang et al., 2 Apr 2026).

The design begins from the observation that existing filesystems fit this environment poorly. Per-host DAX filesystems such as ext4-dax, NOVA, PMFS, and tmpfs preserve separate per-host metadata and caching state even when backed by the same physical memory, thereby duplicating caches and negating memory pooling. Distributed filesystems such as NFS, CephFS, Lustre, Orion, and Assise rely on message passing and server-mediated coordination, introducing microsecond-scale protocol costs before accesses that could otherwise be nanosecond-scale loads and stores. FamFS allows multi-host mounting of CXL-backed storage, but it uses a single-master model in which only one host can create, write, or delete while others replay metadata logs (Wang et al., 2 Apr 2026).

In broader DAX literature, direct access is usually framed as bypassing the page cache so that file contents in persistent memory are reached through direct mappings and ordinary load/store instructions. That reduces software overhead, but it does not by itself depart from a CPU-centric execution model; computation still occurs on host processors even when data is byte-addressable and page-cache bypassed (Dubeyko, 2019). DaxFS differs by treating shared coherent memory itself as the filesystem substrate for concurrent multi-host coordination.

A common misconception is that DaxFS is merely another DAX-enabled filesystem. More precisely, it is a CXL-native shared filesystem whose design premise is that cross-host coordination can be reduced to a single primitive—CAS—because the interconnect itself supplies coherent atomics. This suggests a distinct filesystem design point: shared memory is treated as storage, not merely as a faster backing medium for an otherwise conventional filesystem.

2. Architectural organization and filesystem modes

DaxFS is implemented as a Linux filesystem module of approximately 2.5 KLoC and is exposed as filesystem type daxfs. It uses the Linux VFS, but all persistent filesystem structures are stored directly in a DAX-mapped shared region, either by mapping a physical memory range with memremap() or by using dma-buf mappings for GPU-aware paths. The filesystem supports mount(2) and fsopen/fsconfig/fsmount, and it uses DAX-style mappings without going through dax_device; instead it maps the underlying physical memory via memremap(MEMREMAP_WB) or dma_buf_vmap() (Wang et al., 2 Apr 2026).

Within the shared region, DaxFS organizes data into a small number of explicit regions: a 4 KB superblock; an optional read-only base image containing a flat inode table and data area; an optional hash overlay that stores writable metadata and data through CAS-based structures; and an optional shared page cache whose slots and metadata also reside in the same DAX region. This supports several deployment modes.

Mode	Layout	Role
Static	[Super] [Base]	Read-only image
Empty	[Super] [Overlay] [PCache]	Writable overlay without base image
Split	[Super] [Base] [Overlay] [PCache]	Read-only base plus writable overlay and shared cache

The phrase “lock-free shared filesystem” has a specific meaning in DaxFS. In the multi-host data path, no kernel spinlocks, mutexes, distributed locks, or central lock managers protect data and metadata resident in CXL memory. Cross-host coordination for overlay updates, pool allocation, free-list management, page-cache state transitions, and eviction is performed solely through 64-bit cmpxchg on shared coherent cache lines, with memory barriers such as smp_wmb() and smp_rmb() to enforce publication and visibility ordering. Local per-host OS structures still use ordinary kernel locking, but those locks are not shared across hosts (Wang et al., 2 Apr 2026).

The access model is correspondingly direct. Reads and mmap resolve to CXL memory without per-host buffer caches, and for overlay-backed data the implementation uses per-inode xarrays local to each host to map offsets to DAX addresses. Read paths coalesce physically contiguous pages to reduce copy overhead, while mmap establishes PFN mappings directly to DAX pages so that userspace loads are issued against the shared memory itself (Wang et al., 2 Apr 2026).

3. CAS-based overlay, metadata model, and correctness discipline

The central writable structure in DaxFS is a CAS-based hash overlay layered over an optional immutable base image. All new or modified data and metadata are represented as overlay entries in a lock-free open-addressing hash table backed by a bump-allocated pool. Each bucket contains a 64-bit state_key, where bit 0 encodes FREE versus USED and bits 63:1 encode a 63-bit logical key, together with a pool_off field pointing to the actual object in the pool. Collisions are resolved by linear probing, and insertion proceeds by attempting a cmpxchg from 0 to (key << 1) | 1 on the target bucket’s state_key (Wang et al., 2 Apr 2026).

The 63-bit key unifies several object classes. Data pages use

$$\text{key} = (\text{ino} \ll 20) \mid \text{pgoff},$$

supporting up to $2^{20}$ pages, or 4 GB, per file. Inode entries use

$(\text{ino} \ll 20) \mid 0xFFFFF,$

directory-list heads use

$(\text{ino} \ll 20) \mid 0xFFFFE,$

and directory entries use a 63-bit FNV-1a(parent_ino, name) hash. The pool stores 32-byte inode entries, 4 KB data pages, directory entries of approximately 280 bytes, and 16-byte directory-list nodes; allocation is via a CAS-protected bump pointer, while each type also maintains a lock-free free list (Wang et al., 2 Apr 2026).

This organization gives DaxFS a simple update protocol. A writer claims a free bucket by CAS, allocates and initializes the corresponding pool object, executes smp_wmb() to guarantee content visibility, and only then publishes the pool_off pointer. Readers perform the inverse ordering: they read pool_off, execute smp_rmb(), and then access the pool object. The intended effect is that any reader observing a nonzero published offset also observes a fully initialized object. On x86 this primarily constrains compiler reordering under TSO; on ARM64 it materializes as architectural fences (Wang et al., 2 Apr 2026).

Directory and inode organization follow the same split design. The base image stores directories as flat arrays of fixed-size 271-byte directory entries containing name length, inode number, parent inode, and an inline name up to 254 bytes. The overlay maintains, for each directory, both a directory-list head entry and hashed individual directory entries; readdir enumerates base plus overlay, with overlay entries overriding base entries and tombstones representing deletions. File size is authoritative in the shared overlay inode entry, and each host refreshes its VFS inode size on the read path with READ_ONCE() rather than via explicit invalidation (Wang et al., 2 Apr 2026).

The paper’s correctness claim is not that conflicts disappear, but that conflicts are reduced to retries rather than inconsistency. If two hosts race to insert the same key, only one can win the FREE-to-USED CAS, and the loser observes a USED bucket with a matching key and treats the event as an update or duplicate insertion. Under well-provisioned load factors, linear probing eventually reaches either a matching key or a free slot. This is lock-free rather than wait-free: some participant always makes progress, but individual threads may retry (Wang et al., 2 Apr 2026).

4. Cooperative shared page cache and MH-clock eviction

When file data resides in a separate backing file, DaxFS uses a shared page cache, or pcache, located in the same DAX region and visible to all hosts. Each page-cache slot carries a 64-bit state_tag word and a ref_bit. The low two bits of state_tag encode FREE, PENDING, or VALID; bits [5:2] hold a 4-bit active-reader refcount from 0 to 15; and bits [63:6] hold a tag

$(\text{ino} \ll 20) \mid \text{pgoff}.$

By packing state, refcount, and tag into one 64-bit word, DaxFS makes slot state transitions CAS-addressable in a single atomic operation (Wang et al., 2 Apr 2026).

Cold-miss handling is coordinated without locks. A host probes up to eight candidate slots. If it finds a VALID slot with a matching tag, it increments the refcount by CAS and returns the slot. If it finds a FREE slot and no valid match, it attempts a CAS from FREE to PENDING for the desired tag. The winner reads the page from the backing file into slot memory with kernel_read and then CASes the slot to VALID. Other hosts that encounter a PENDING slot for the same tag spin until the state becomes VALID, thereby avoiding duplicate fills (Wang et al., 2 Apr 2026).

Eviction is handled by a novel multi-host CLOCK algorithm, MH-clock. In each host’s local probe window, victim selection first seeks slots with state = VALID, ref_bit = 0, and refcount = 0; if such a slot exists, the host CASes its state_tag to FREE. If all candidate slots are hot, indicated by ref_bit = 1, the host clears those bits with plain stores, yields briefly, and rescans. If no cold slot appears, it force-evicts the first VALID slot whose refcount is zero, irrespective of ref_bit. Because eviction is constrained by refcount == 0, slots currently in use by any host are not reclaimed (Wang et al., 2 Apr 2026).

MH-clock also includes a background sweep driven by a shared atomic evict_hand. A host may CAS-acquire ownership of a window of 64 slots and clear ref_bit values in that interval; hosts that lose the CAS skip the sweep. The effect is a distributed age-demotion mechanism that preserves the “second chance” behavior of classical CLOCK while avoiding centralized eviction control (Wang et al., 2 Apr 2026).

The same coordination model was extended experimentally to GPUs. DaxFS can expose its DAX region via DAXFS_IOC_GET_DMABUF, allowing the GPU driver to map the shared region as pinned host memory. GPU threads can perform pcache lookups by reading slot metadata and issuing atomicCAS for slow paths. On an RTX 5090 over PCIe 5.0, independent-slot 64-bit atomicCAS reached approximately 11.7 Mops/s at 512 threads, close to the reported PCIe 5.0 AtomicOp bandwidth limit of about 11.5 Mops/s. This does not constitute full GPU write support, which remains unimplemented, but it demonstrates that the atomics-only coordination model extends beyond CPUs (Wang et al., 2 Apr 2026).

5. Implementation, validation, and measured performance

The evaluation of DaxFS combines emulation-based correctness testing with DRAM-backed performance experiments. Multi-host correctness was validated in a patched QEMU 10.0 CXL 3.0 environment in which two virtual machines shared a GFAM-like region and CXL atomic operations were forwarded over TCP. This emulation is orders of magnitude slower than DRAM, but it exercises the CAS protocol under real concurrency. In those tests, DaxFS maintained more than 99% CAS “accuracy” under cross-host contention, and overlay insertions exhibited no lost updates even when both hosts created entries concurrently (Wang et al., 2 Apr 2026).

Single-host throughput experiments used DRAM-backed DAX memory as a stand-in for CXL memory. Platform A combined dual-socket Intel Xeon processors, 512 GB DDR5, and an NVIDIA RTX 5090 with a 512 MB DaxFS region allocated via CMA; Platform B used an Intel Xeon Gold 5418Y for comparisons against ext4-dax and tmpfs. The benchmark methodology used fio 3.x with synchronous I/O, 64 MB files, and filesystem reformatting between write experiments; the overlay was configured with a 400 MB pool and 65,536 buckets (Wang et al., 2 Apr 2026).

Several results are structurally important. For sequential writes, DaxFS exceeded tmpfs across all write workloads, including 1,730 MiB/s versus 1,362 MiB/s for a 4 KB first write and 2,783 MiB/s versus 2,560 MiB/s for a 1 MB rewrite. For random writes, the most prominent result was 4 KB random write throughput with four threads: 4,830 MiB/s for DaxFS versus 1,803 MiB/s for tmpfs, or 2.68× higher throughput. For random reads, DaxFS reached 15,052 MiB/s versus 12,800 MiB/s for tmpfs at 64 KB with four threads, a 1.18× improvement. Against ext4-dax on Platform B, DaxFS was 6.1× faster in 16 MB read latency, at 274.2 ms versus 1,660.7 ms, although tmpfs remained faster on some large sequential read cases (Wang et al., 2 Apr 2026).

Workload	DaxFS result	Baseline comparison
Cross-host CAS under contention	\>99% CAS accuracy	No lost updates in overlay insertions
4 KB random write, 4 threads	4,830 MiB/s	tmpfs: 1,803 MiB/s, 2.68× lower
64 KB random read, 4 threads	15,052 MiB/s	tmpfs: 12,800 MiB/s
16 MB read latency	274.2 ms	ext4-dax: 1,660.7 ms

The paper attributes these behaviors to the avoidance of per-page faults, folio accounting, page-cache locks, and inode xarray contention in paths where DaxFS instead uses pre-allocated DAX pages, lock-free overlay lookups, and direct pointer-based copies. The counterpoint is that DaxFS may pay per-page overlay resolution overhead on large sequential reads and may scan both base and overlay during readdir, where it was slower than both tmpfs and ext4-dax (Wang et al., 2 Apr 2026).

6. Limitations, related concerns, and significance

DaxFS makes several explicit trade-offs. Its directory representation is flat, with $O(n)$ lookup behavior that is described as acceptable for modest sizes below 10K entries but expected to degrade for very large directories. The overlay hash table has a fixed size chosen at format time and does not support dynamic resizing. The pool does not implement compaction, tolerating fragmentation while relying on free-list reuse. Persistence assumes ADR or eADR and does not issue explicit clflush or clwb; no mechanism is provided for multi-host crash recovery, reconstruction after partial writes, or host failure in mid-operation. POSIX support is incomplete, with no mknod, FIFOs, sockets, or extended attributes, and filename length is capped at 255 characters (Wang et al., 2 Apr 2026).

A second misconception is that absence of journaling in the data path implies a general solution to durability and recovery. The paper states the opposite: the current prototype omits those mechanisms and treats crash consistency, persistence ordering on non-ADR systems, and full failure handling as future work. This suggests that DaxFS should presently be understood as a lock-free coordination and caching architecture rather than a complete answer to all persistence semantics in shared disaggregated memory.

There is also a broader systems concern inherited from DAX itself. Direct mappings make file access fast because subsequent operations are ordinary loads and stores, but that same property weakens syscall-centric observability. Prior work on hardware-assisted auditing for DAX-hosted filesystems argues that after the initial mmap, read and write activity within mapped regions becomes largely invisible to conventional OS-level auditing and provenance mechanisms (Ye, 2021). DaxFS uses the same direct-access style for shared memory; this suggests that fine-grained auditability, provenance, and forensic logging remain open concerns for practical deployments, particularly in multi-host or accelerator-sharing environments.

Within the filesystem literature, DaxFS therefore represents a specific architectural thesis. Classical DAX reduces page-cache overhead but remains within a host-centric access model (Dubeyko, 2019). DaxFS goes further by assuming a coherent shared-memory fabric and collapsing multi-host filesystem coordination to CAS on that fabric. Its significance lies less in feature completeness than in the demonstration that coherent atomics can replace distributed locks, centralized coordinators, and per-host caches for an important class of shared-memory filesystems.