FalconFS: Distributed File System

• FalconFS is a distributed file system engineered for deep learning pipelines, featuring a stateless-client model and one-hop metadata resolution.
• It utilizes hybrid metadata indexing, lazy namespace replication, and concurrent request merging to deliver enhanced scalability and performance.
• Its real-world deployment in large-scale systems, such as Huawei's autonomous driving clusters, demonstrates reduced client memory overhead and rapid file operations.

FalconFS is a distributed file system (DFS) specifically engineered to support the high-throughput, metadata-intensive demands of large-scale deep learning pipelines. Departing fundamentally from the prevalent paradigm of client-side metadata caching, FalconFS employs a stateless-client architecture, offloading all path resolution and metadata management to an optimized, server-side infrastructure. This approach is demonstrated to alleviate substantial memory overhead on client nodes while simultaneously enabling rapid, one-hop file operations—even within directory trees of immense size and complexity. FalconFS combines hybrid metadata indexing, lazy namespace replication, concurrent metadata request merging, and kernel-level integration via the VFS shortcut, yielding significant improvements in performance, scalability, and operational simplicity in production environments such as Huawei's autonomous driving system clusters with 10,000 NPUs (Xu et al., 14 Jul 2025).

1. Stateless-Client Architecture and System Design

FalconFS is constructed around a stateless-client architecture, representing a departure from traditional DFS designs such as CephFS and Lustre. In conventional systems, client nodes maintain local caches of directory entries (dentries) and other metadata, improving path lookup times at the cost of significant memory consumption and cache coherence complexity. FalconFS eschews all client-side metadata state; instead, each file access request—including full file paths—is transmitted directly to one of several specialized metadata servers termed MNODES.

Each MNODE is responsible for a partition of file inodes but also maintains a replica of the entire directory namespace. Directory entries are thus replicated across all MNODES, ensuring that any metadata server can independently resolve any path without additional network hops. This replication is maintained in a lazy, on-demand manner, minimizing synchronization costs.

The primary components of the FalconFS system are:

• Client Module: Implemented as a kernel module, this intercepts filesystem calls and short-circuits the traditional Linux VFS path walk via the VFS shortcut mechanism.
• MNODES: Metadata servers running a modified transactional engine (e.g., with PostgreSQL foundations, B-link trees, write-ahead logging) that manage inodes and a lazily updated namespace replica.
• Coordinator: Handles namespace mutations, monitors load balance, and manages an exception table for load skew mitigation.
• File Store: Manages actual user data, indexing file blocks via hashing.

This design enables one-hop metadata resolution even in billion-directory environments, eliminating the “lookup tax” of multiple remote lookups per file operation.

2. Metadata Management Techniques

FalconFS incorporates several key innovations in metadata management to address the unique requirements of deep learning workflows:

A. Hybrid Metadata Indexing

• Filename Hashing: By default, FalconFS uses filename hashing to map file inodes to specific MNODES, eliminating dependence on local caches and enabling fast, cache-free file lookup.
• Redirection Mechanisms: To manage hot spots and mitigate hash-based load imbalances, two redirection strategies are employed:
  • Path-walk redirection dynamically includes the parent directory ID in the hash function for more distributed placement.
  • Overriding redirection allows explicit reassignment of problematic filenames to less-loaded MNODES.
• Exception Table: These redirections are coordinated via a globally synchronized exception table among clients, MNODES, and the coordinator.

B. Lazy Namespace Replication

• Directory entry replicas are not eagerly synchronized. Instead, missing entries discovered during path resolution are retrieved on demand from the owning MNODE.
• Namespace-modifying operations (e.g., deletes, renames) use an invalidation-based (cache coherence–like) protocol to inform MNODES of changes, maintaining correctness without persistent locking.

C. Concurrent Request Merging

• MNODES improve throughput by batching concurrent metadata requests, executing them as a single transaction. This reduces per-operation lock overhead and amortizes write-ahead log flushes, ideal for highly parallel deep learning job execution.

D. VFS Shortcut Deployment

• The FalconFS client module intercepts and emulates VFS lookups of intermediate directories with permissive “fake” attributes, only invoking real remote metadata validations upon actual file operations (leveraging Linux’s d_revalidate call). This enables seamless integration with Linux environments without wholesale kernel changes.

3. Performance Characteristics

Comparative evaluations of FalconFS against CephFS and Lustre highlight the specific performance benefits of FalconFS in data-intensive, metadata-heavy environments:

• Small-file I/O throughput: up to 5.72× higher than CephFS.
• Deep learning model training throughput: up to 11.81× improvement over CephFS, attributed primarily to rapid and scalable metadata operations during random file access.
• Latency: While some increase in per-request latency is introduced due to batching, aggregate system throughput and efficiency are notably superior—particularly important under constrained client memory conditions.

The reported performance gains directly reduce the operational overhead associated with client caching and network-amplified metadata lookups that typify classical DFS deployments in deep learning pipelines.

4. Real-World Deployment and Applications

FalconFS has been extensively deployed in large-scale, production-grade environments:

• Huawei autonomous driving platform: Running on 10,000 NPUs for tasks such as data labeling and model training, FalconFS is tasked with managing directory structures comprising billions of directories and hundreds of billions of files.
• In such scenarios, the removal of client-side caching dramatically lowers memory consumption on compute nodes, enables one-hop metadata resolution for any file operation, and preserves performance at scale.
• Operational feedback notes that these properties are essential for maintaining high levels of parallelism and low latency, both critical to deep learning training throughput and system responsiveness.

5. Scalability and Load Balancing Mechanisms

FalconFS achieves scalability via several mechanisms tailored to the nuances of modern deep learning system workloads:

• Namespace Replication: Each MNODE maintains a full, albeit lazily updated, copy of the directory namespace, allowing for stateless and immediate path resolution without inter-server negotiation.
• Hybrid Indexing and Load Balancing: The dual-mode indexing, coupled with the coordinator’s real-time load monitoring and dynamic reallocation (via the exception table), ensures no MNODE is overloaded by hot keys or pathological namespace distributions.
• Batch Processing: The request batching mechanism on MNODES efficiently handles thousands of concurrent deep learning training tasks and random file access streams.
• Stateless Clients: Absence of client-side metadata simplifies scaling for compute clusters—new clients can be simply added without jeopardizing system balance or requiring cache warm-up.

6. Implications and Context within Distributed File Systems

FalconFS represents a shift from conventionally client-driven performance optimization to a fully server-centric metadata scheme. This design is explicitly motivated by the operational characteristics of deep learning data pipelines, in which gigantic directory trees and highly parallel, random access patterns render cache-based techniques impractical or counterproductive. The architectural principles—stateless clients, lazy but holistic metadata replication, adaptive load balancing—mirror advances in distributed systems seeking to minimize coordination overhead while ensuring locality and resilience in the face of skewed workloads.

A plausible implication is that as deep learning training and related high-throughput applications scale further, the stateless-client and distributed metadata model exemplified by FalconFS may become increasingly relevant for broader classes of large-scale, data-intensive computing systems.