ALEX: An Updatable Adaptive Learned Index (1905.08898v2)

Abstract: Recent work on "learned indexes" has changed the way we look at the decades-old field of DBMS indexing. The key idea is that indexes can be thought of as "models" that predict the position of a key in a dataset. Indexes can, thus, be learned. The original work by Kraska et al. shows that a learned index beats a B+Tree by a factor of up to three in search time and by an order of magnitude in memory footprint. However, it is limited to static, read-only workloads. In this paper, we present a new learned index called ALEX which addresses practical issues that arise when implementing learned indexes for workloads that contain a mix of point lookups, short range queries, inserts, updates, and deletes. ALEX effectively combines the core insights from learned indexes with proven storage and indexing techniques to achieve high performance and low memory footprint. On read-only workloads, ALEX beats the learned index from Kraska et al. by up to 2.2X on performance with up to 15X smaller index size. Across the spectrum of read-write workloads, ALEX beats B+Trees by up to 4.1X while never performing worse, with up to 2000X smaller index size. We believe ALEX presents a key step towards making learned indexes practical for a broader class of database workloads with dynamic updates.

• The paper introduces ALEX, which combines machine learning with a gapped array structure to optimize dynamic database indexing.
• The paper presents a cost-based mechanism that maintains model accuracy by selectively retraining and expanding index trees as data evolves.
• The paper demonstrates that ALEX outperforms traditional structures like B+Trees, achieving higher throughput and lower memory footprints in complex workloads.

ALEX: An Updatable Adaptive Learned Index

The paper presents ALEX, an innovative and practical solution for dynamic workloads in database indexing, emphasizing its efficacy across read-write operations and its foundational basis in the theory of learned indexes. This represents a significant development in database management systems, as it evolves beyond the limitations of the original learned index model introduced by Kraska et al., which was constrained to static, read-only data scenarios.

Core Contributions

ALEX leverages the core insight of learned indexes—viewing an index as a model predicting key positions. By integrating traditional storage techniques with machine learning models, ALEX adapts dynamically to various workloads encompassing point lookups, inserts, updates, deletes, and range queries.

1. Gapped Array Data Structure: ALEX employs a gapped array layout in its data nodes, contrasting the traditional densely packed arrays. This structure introduces strategically placed gaps between elements, enabling efficient insert operations by amortizing the cost of key shifts and aiding accurate model-based inserts.
2. Dynamic Model Accuracy Maintenance: To maintain model accuracy with changing data distributions, ALEX integrates a cost-based mechanism that selectively re-trains models or expands the tree structure. This is crucial for supporting dynamic data patterns without necessitating significant manual tuning.
3. Search and Insert Strategies: ALEX utilizes exponential search starting from the predicted key position, which is especially beneficial when models yield precise predictions. This method surpasses binary search in performance, given the models’ accuracy, which ALEX ensures via model-based insertions.
4. Evaluative Cost Modeling: ALEX uses predictive models to evaluate the cost of operations, guiding expansion and split decisions within nodes. These cost models ensure that changes to the index structure respond to workload dynamics, optimizing for both time and space efficiency.

Performance and Index Efficiency

Through extensive evaluation against conventional B+Tree, model-enhanced B+Tree, and other state-of-the-art indexing methods like the Adaptive Radix Tree, ALEX demonstrates superior performance. It achieves higher throughput in read-heavy, write-heavy, and mixed workloads, and retains a much smaller memory footprint across benchmarks. Notably, ALEX provides up to 2.2 times higher throughput than previous learned indexes with up to 15 times smaller index size on read-only tasks, while ensuring up to 36000 times size reduction relative to some traditional structures on dynamic workloads.

Theoretical and Practical Implications

Theoretical contributions include formally bounding the depth and cost complexities associated with ALEX’s operations. Practically, ALEX showcases robustness to distribution shifts and large data scaling, presenting significant implications for OLTP systems. This positions ALEX as a compelling choice for real-time applications that require rapid adaptation to data changes.

The practical viability of ALEX lies not only in its performance gains but also in its adaptability. By automatically adjusting its structures through cost models without requiring exhaustive data-specific tuning, ALEX reduces operational overhead in live systems.

Future Development in AI-Enhanced Indexing

Future avenues for AI in databases might explore further enhancements in indexing precision and speed. Developments could focus on refining adaptive learning techniques and expanding indexing models that can intuitively handle even more complex workloads and data types.

In conclusion, ALEX represents an advancement in index design, effectively combining machine learning with database systems' foundational constructs to tackle a broader spectrum of dynamic operations. As database environments continue to grow in complexity, innovations like ALEX will increasingly shape the landscape of efficient data management and retrieval.