eCP-FS: File-Structured ANN Index

• eCP-FS is a file-structured implementation of the hierarchical, cluster-based eCP index, mapping its data to a transparent filesystem layout using Zarr.
• It supports incremental, stateful retrieval by maintaining persistent query state and enabling lazy loading with controlled caching under fixed memory budgets.
• While its cold-start performance is slower due to extensive file I/O, eCP-FS excels in resource-constrained, multi-index environments by offering language-agnostic, inspectable access.

eCP-FS is a file-structured implementation of eCP, a hierarchical, cluster-based disk-oriented approximate nearest-neighbor (ANN) index. Rather than storing the index as a conventional opaque serialized binary blob, eCP-FS represents the internal hierarchy as a transparent file and folder structure using Zarr groups and arrays. The design goal is not to replace eCP’s broad indexing logic, but to make the index easily readable for any programming language and even human-readable, while preserving disk-based operation and enabling incremental retrieval under a tightly controllable memory budget. The resulting trade-off is explicit: eCP-FS is slower than highly optimized ANN systems, but it offers readability, inspectability, language-agnostic access, and very low controllable memory usage, especially in memory-constrained or multi-index settings (Khan et al., 29 Jul 2025).

1. Definition, lineage, and motivation

eCP-FS is defined as a file-based implementation of eCP. The underlying eCP method is a hierarchical cluster-based index in which cluster leaders are selected from the data and the structure is built top-down to speed indexing and support disk-based storage. eCP-FS retains that broad hierarchical organization, but changes the storage model: the ANN hierarchy is exposed as a file structure rather than hidden inside a bespoke binary serialization (Khan et al., 29 Jul 2025).

The proposal is motivated by two problems identified for modern ANN deployments. The first is an opacity problem: if the structure is embedded in code or a serialized binary file, it is difficult to visualize, analyze, or mine the index structure. The second is resource pressure: modern analytical pipelines often run multiple heavy components simultaneously—several ANN indexes, embedding models, LLMs/VLMs, and related services—so memory becomes a scarce shared resource. A disk-based, incremental, cacheable design can reduce this pressure by avoiding full in-memory loading.

Within this framing, eCP-FS is not presented as a new ANN algorithm from scratch. Its distinguishing feature is the decision to map the internal data structure to a file structure. This makes the hierarchy more transparent and more accessible from heterogeneous software environments, but it also makes the serialized representation verbose, which introduces search overhead. The central question of the work is therefore not whether the representation is cleaner, but how severe the resulting performance penalty is and under what operating conditions the trade-off is favorable.

2. File-structured representation and storage model

The file layout mirrors the conceptual hierarchy of the eCP index. The structure includes:

• info for global metadata such as maximum level and metric
• rep_embeddings and rep_item_ids for representative items used to build the hierarchy top-down
• index_root for embeddings and IDs at the first level
• lvl_[0..L] for groups corresponding to hierarchy levels, where each node group contains embeddings and IDs pointing to the next level

This arrangement is the core of the file-structure idea: the ANN index is represented as nested file groups, so metadata, representatives, per-level nodes, embeddings, and item IDs are all exposed as concrete file-system-like entities rather than hidden inside an opaque index image (Khan et al., 29 Jul 2025).

The implementation uses Zarr. The paper attributes several roles to this choice: concurrent access, language neutrality, longevity/support, and flexible storage of arrays and metadata. These properties are central to the intended use case. A conventional serialized binary index generally requires specialized parsing code and tight coupling to one implementation, whereas a Zarr-based layout can be accessed from multiple languages and environments and can be extended with embedded metadata, alternative vector representations, or additional feature data.

A further design element is lazy loading and caching. At startup, eCP-FS reads only the info group and the index_root group, and constructs node objects for each level without loading all node data. Node contents are then loaded on demand during search, cached for reuse, and managed under a fixed memory budget. The proposed controls include a maximum number of nodes allowed in RAM, an LRU eviction policy, and optional prefetching up to a specified level using background threads. This gives the system a tunable memory-performance trade-off and is one of the main reasons the implementation is positioned for resource-constrained deployment.

3. Search procedure and incremental retrieval

A major contribution beyond the storage layout is the shift from standard eCP traversal to incremental retrieval. The original eCP search follows the best $b$ nodes at a level before proceeding. eCP-FS instead uses a single priority queue $T$ and an output list $I$, expanding the globally best available node regardless of level. This makes the search state persistent and resumable (Khan et al., 29 Jul 2025).

Each query has a state object

$Q = \{q, T, I\},$

where $q$ is the query vector, $T$ is the priority queue of pending nodes, and $I$ is the list of retrieved items. These states are stored in a query-state table $QS$, and each query receives a query ID.

The described workflow is as follows. A new search creates

$Q = \{q, T=\emptyset, I=\emptyset\},$

appends it to $QS$, assigns

$T$0

runs IncrementalSearch(q_id, k, b, mx_inc, E), and returns $T$1. A follow-up request for additional results checks whether $T$2 already contains enough items; if so, it returns the next $T$3, and if not, it resumes search by calling IncrementalSearch again.

The mechanics of incremental search are explicitly stateful. If $T$4 is empty, the algorithm computes distances from the query to root children and pushes them into $T$5. It then pops the best node from $T$6. If the node is internal, its children are pushed into $T$7; if it is a leaf node, its children or items are added into $T$8. After $T$9 leaf nodes have been examined, the algorithm checks whether enough results exist. If not, it can double $I$0 and continue, up to a maximum number of increments $I$1.

This organization supports several behaviors that are difficult to realize efficiently in stateless ANN interfaces: resuming a query later, returning more results on demand, and filtering or exclusion during retrieval. The paper also states an important caveat. Because traversal is governed by a global priority queue rather than level-local queues, the new method no longer strictly limits internal-node exploration to $I$2 per level; it may explore any number of internal nodes while searching for the best $I$3 clusters. The authors note that level-specific queues could restore stricter bounds, but they did not implement that variant.

4. Hierarchical sizing formulas and traversal cost

The paper gives explicit sizing formulas for the eCP hierarchy and uses them to explain the search cost model. Given:

• $I$4: number of items
• $I$5: feature-vector size in bytes
• $I$6: desired cluster size
• $I$7: index depth

the number of cluster leaders is computed as

$I$8

To reduce traversal cost, the number of internal subsets per level is chosen as

$I$9

If only the single best branch is followed, the traversal cost is approximately

$Q = \{q, T, I\},$0

distance calculations (Khan et al., 29 Jul 2025).

The paper’s worked example uses:

• $Q = \{q, T, I\},$1
• CLIP embeddings of dimension $Q = \{q, T, I\},$2
• float16 storage, so

$Q = \{q, T, I\},$3

• target cluster size

$Q = \{q, T, I\},$4

• depth

$Q = \{q, T, I\},$5

From these values,

$Q = \{q, T, I\},$6

so each cluster should contain about 57 descriptors, and

$Q = \{q, T, I\},$7

The level branching factor becomes

$Q = \{q, T, I\},$8

The resulting hierarchy is described as 26 nodes at the top level, each with 26 second-level children, each with 26 leaf clusters, with each leaf holding about 57 descriptors. The estimated cost is then

$Q = \{q, T, I\},$9

distance computations on average.

For expanded search with parameter $q$0, the paper generalizes to an average query cost

$q$1

The text notes a formatting glitch around this expression, but presents it as the intended form. Taken together, these formulas show that eCP-FS inherits eCP’s hierarchical cost logic even though its storage and retrieval model differ substantially from conventional serialized ANN indexes.

5. Empirical behavior: latency, caching, and memory footprint

The evaluation compares eCP-FS with IVF, HNSW, and DiskANN on LSC24, V3C1, V3C with PCA-reduced dimensions $q$2 and $q$3, and V3C at full $q$4 dimensions only for eCP-FS. The benchmark platform is a laptop with Windows 11, 16 GB RAM, NVMe SSD, and Intel i9-12900H (Khan et al., 29 Jul 2025).

For single-query latency, eCP-FS is slower than highly optimized in-memory indexes, especially on cold disk access. The paper gives the following examples:

Dataset	Reported latency figures	Interpretation
LSC24	IVF: memory latency $q$5 s; HNSW: $q$6 s; DiskANN: disk latency $q$7 s; eCP-FS: disk $q$8 s, memory $q$9 s, workload $T$0 s $T$1	Cold eCP-FS is slow on disk; once hot in memory it becomes much more competitive
V3C1	eCP-FS: disk $T$2 s, memory $T$3 s, workload $T$4 s $T$5	Many file opens and closes dominate cold-start cost
V3C (1152)	eCP-FS: disk $T$6 s, memory $T$7 s, workload $T$8 s $T$9	Full-dimension disk access further amplifies overhead

The paper’s interpretation is precise. Cold eCP-FS is slow on disk because it must open and close many files. Once hot in memory, it becomes much more competitive; after caching, it is stated to be roughly within a factor of about 2 of IVF in some cases. By contrast, DiskANN is much faster on disk because it uses a small number of serialized files, which is more efficient than thousands of file opens and closes.

The evaluation is more favorable to eCP-FS for incremental-search latency, where persistent query state changes the comparison. Table 3 is summarized with examples such as:

• LSC24: IVF $I$0 s, HNSW $I$1 s, DiskANN $I$2 s, eCP-FS $I$3 s $I$4
• V3C1: IVF $I$5 s, HNSW $I$6 s, DiskANN $I$7 s, eCP-FS $I$8 s $I$9
• V3C (720): eCP-FS $QS$0 s $QS$1
• V3C (1152): eCP-FS $QS$2 s $QS$3

The reason given is structural: other systems do not preserve query state, so each follow-up request effectively repeats search work, whereas eCP-FS stores $QS$4 and $QS$5, so subsequent requests may only need to return items already in memory.

The paper also treats memory footprint as a central axis of comparison. IVF and HNSW load the whole dataset into memory. DiskANN keeps memory lower via a compact graph and PQ-compressed vectors. eCP-FS does not load the entire dataset and does not rely on compression for the stored data. Instead, raw data is stored in original form on disk; when loaded, the Rust version converts float16 embeddings to float32; and memory use is controlled by caching and eviction. The paper warns that uncontrolled caching can exceed available memory, but states that with LRU-style limits or caching off, the footprint can be kept minimal.

6. Comparative position, applicability, and limitations

The paper characterizes eCP-FS as advantageous under several specific conditions. It is favorable when memory is scarce, because it does not require the whole index in RAM and supports selective node loading. It is also favorable when multiple ANN indexes or heavy models coexist, because it minimizes memory competition with other components. A third favorable setting is incremental or resumable retrieval, including live search, browsing, and interactive systems in which users repeatedly ask for more results. The same design is also beneficial when transparency and inspectability matter, because the file structure is easier to understand, debug, and analyze, and when language-agnostic integration is needed, because Zarr can be accessed from different environments (Khan et al., 29 Jul 2025).

The disadvantages follow directly from the same design choices. When raw latency is the top priority, IVF and HNSW are faster in memory, and DiskANN is more efficient on disk because it relies on fewer, more controlled serialized files. When cold-start performance matters, eCP-FS incurs high disk I/O overhead because uncached traversal requires many file operations. When the application does not need resumable search, the stateful machinery offers less benefit. The current incremental-search formulation also has a formal limitation: it does not preserve a strict per-level bound on internal-node exploration. Finally, for extremely large datasets with poor caching discipline, loaded nodes converted to float32 can still consume substantial memory.

These trade-offs lead to a clear positioning. eCP-FS is a transparency-first, disk-based ANN index. Its distinctive value lies not in maximum speed, but in combining a readable and extensible storage structure with minimal memory footprint and stateful incremental retrieval. The paper’s empirical conclusion is correspondingly nuanced: if the sole objective is maximum speed, optimized in-memory ANN systems remain preferable; if the objective is efficient disk-based retrieval through compact serialized files, DiskANN is stronger on raw disk access; but if the objective is a readable ANN structure, very low controllable memory usage, and resumable search, eCP-FS is particularly well suited to multi-index, resource-constrained, or long-running interactive search settings.