DaiSy: A Library for Scalable Data Series Similarity Search

Abstract: Exact similarity search over large collections of data series is a fundamental operation in modern applications, yet existing solutions are often fragmented, specialized, or tailored to specific execution environments. In this paper, we present DaiSy, a unified library for exact data series similarity search that integrates multiple state-of-the-art algorithms within a single, coherent framework. DaiSy is the first library to support exact similarity search across diverse execution environments, including implementations for disk-based, in-memory, GPU-accelerated, and distributed scalable similarity search. Although designed for data series, DaiSy is also directly applicable to exact similarity search over vector data, enabling its use in a broader range of applications. The library supports interfaces in both C++ and Python, enabling users to easily integrate its functionality into a variety of tasks. DaiSy is open-sourced and available at: https://github.com/MChatzakis/DaiSy.

• The paper introduces DaiSy, a unified library for exact similarity search over massive data series and vector data.
• It integrates state-of-the-art algorithms (ParIS+, MESSI, SING, Odyssey) optimized for disk, in-memory, GPU, and distributed systems.
• Rigorous benchmarks on 100M-scale datasets show up to 14× speedup over baseline methods, ensuring practical performance for real-world analytics.

DaiSy: A Unified Library for Scalable, Exact Data Series Similarity Search

Motivation and Problem Statement

Exact similarity search over massive collections of data series remains a fundamental operation in analytics across domains such as finance, astrophysics, neuroinformatics, seismology, and engineering. With the proliferation of high-dimensional data, the requirement for scalable, exact (non-approximate) search methods, able to operate across diverse hardware and resource environments, is increasingly critical. Existing solutions are fragmented, specialized, or limited to specific execution contexts, impeding practical integration and adoption. DaiSy addresses this gap by providing a single, unified library that consolidates state-of-the-art algorithms for exact data series similarity search—including versatile support for both data series and general vector data—across disk-based, in-memory, GPU-accelerated, and distributed systems (2603.27719).

Library Architecture and System Design

DaiSy's architecture follows strict modularity and extensibility principles, enforcing separation of concerns along three axes: distance computation (exact and lower-bound), data access (adapter-based, agnostic to physical layout), and execution strategy (independent search algorithms). The C++ core layer encapsulates foundational primitives and is mirrored by a Python interface, ensuring API uniformity.

Figure 1: Component diagram illustrating DaiSy’s modular, layered architecture separating core primitives, indexing, and execution models.

The index layer encapsulates an iSAX-based structure, facilitating hierarchical discretization and representation. It supports structured access to data series without constraining search strategy. The similarity search layer adopts execution model–centric abstraction, exposing algorithms optimized for disk-based (ParIS+), in-memory (MESSI), GPU-based (SING), and distributed (Odyssey) environments, each guaranteeing exact answers. Importantly, these methods are interchangeable and can be configured via a unified interface, with hyperparameters defaulted to recommended values from prior research.

Supported Algorithms and Execution Models

DaiSy integrates four state-of-the-art algorithms, each the best-in-class for its execution environment:

• ParIS+ (Disk-Based): Efficient index-guided similarity search when datasets exceed RAM capacity.
• MESSI (In-Memory): Parallel, index-guided in-memory search, combining exploration and refinement phases.
• SING (GPU-Accelerated): Leverages GPU hardware for scalable acceleration, with in-memory indexing and offloaded computation.
• Odyssey (Distributed): MPI-based, multi-node, distributed-memory algorithm suited for petabyte-scale datasets.

In addition, DaiSy provides brute-force and lower-bound–enhanced (LbBruteforce) exhaustive search implementations for baseline evaluation.

Algorithm Selection and Practical Usage

DaiSy automates algorithm selection via decision logic based on dataset size and resource availability (RAM, GPU, distributed cluster). When the dataset fits in RAM, Odyssey is prioritized in distributed settings, SING is used if GPU acceleration is present, and MESSI is chosen otherwise; ParIS+ is used for disk-based cases.

Figure 2: Decision tree depicting DaiSy’s algorithm selection process based on data size and hardware resources.

The API exposes standardized calls: buildIndex for initialization, and searchIndex for querying. Both C++ and Python interfaces are provided, facilitating adoption in academic and industrial settings.

Performance Evaluation and Numerical Results

Rigorous benchmarking demonstrates DaiSy's superiority for exact vector search, outperforming FAISS-IndexFlat (the de facto baseline for vector search) by substantial margins. On 100 million–scale datasets (Deep100M, Seismic100M), DaiSy-MESSI achieves up to $12\times$ speedup for deep image embeddings and $14\times$ speedup for seismic data series. Performance improves further as thread count increases.

Figure 3: Query answer time (seconds) for 100 queries on Deep100M as $k$ varies, showing $12\times$ speedup of DaiSy-MESSI over FAISS-IndexFlat.

These results underscore DaiSy’s practical impact, making it the solution of choice for applications requiring exact answers—both for real-world analytics and for generating ground-truth in approximate search evaluations.

Implications and Future Directions

DaiSy's unified, scalable framework sets the stage for practical adoption in domains requiring exact series and vector search at scale, including scientific discovery, anomaly detection, and high-precision retrieval. The architectural separation ensures extensibility for future advancements, including incorporation of algorithms from alternate summarization families (e.g., APCA, Hercules), automatic hyperparameter optimization (e.g., Bayesian techniques), and support for emerging use cases such as subsequence similarity search, streaming data analytics, and progressive/early termination methods. The open-source release and API uniformity further promote reproducibility and system integration.

On the theoretical front, DaiSy's abstraction of index management from execution strategy offers a template for future indexing architectures, facilitating algorithmic innovation without API fragmentation. Practical implications extend to benchmarking pipelines for approximate vector search, where DaiSy can generate exact ground-truth efficiently—a capability previously limited by naive brute-force baselines.

Conclusion

DaiSy establishes itself as a comprehensive library for scalable, exact similarity search, integrating leading algorithms into a unified, extensible framework. Its modular architecture, superior performance, and broad applicability to both data series and vectors position it as a foundational tool for future research and application development in large-scale analytics (2603.27719).