RadixSpline: Efficient Learned Index

• RadixSpline is a learned index structure that uses piecewise-linear spline interpolation with error-bound guarantees for efficient key lookups.
• It combines a flat radix table for rapid segment localization, optimizing build time, space, and lookup performance compared to traditional methods.
• By tuning the error bound and radix bits, RadixSpline achieves a balanced trade-off between memory usage and latency, benefiting LSM-tree systems and similar applications.

RadixSpline is a learned index structure designed to efficiently approximate position lookups over sorted arrays of key/position pairs. It combines a piecewise-linear, error-bounded spline with a flat radix table for rapid segment localization, providing competitive size and lookup performance compared to state-of-the-art techniques while requiring only a single construction pass and two tunable parameters (Kipf et al., 2020).

1. Formal Description

A RadixSpline index is built over a sorted dataset $$D = [(k_0, p_0), (k_1, p_1), \ldots, (k_{N-1}, p_{N-1})]$$, where each $k_i$ is a key and $p_i$ its position.

• The spline $S(\cdot)$ consists of $M$ “knots”: $(x_0, y_0), (x_1, y_1), \ldots, (x_{M-1}, y_{M-1})$, with $x_j$ strictly increasing and $y_j$ denoting positions.
• $S(x)$ interpolates linearly between consecutive knots:

$$S(x) = y_i + \frac{y_{i+1}-y_i}{x_{i+1}-x_i} (x - x_i), \quad \text{for } x_i \le x \le x_{i+1}.$$

• For every key, $|S(k_i) - p_i| \leq E$, enforcing the error bound $E$.

A flat radix table $T[0\ldots2^r]$ maps the $r$ most significant bits of the keys (with any fixed common prefix removed) to knot intervals:

• For $b = \text{top}_r\_\text{bits}(x)$,
  • $T[b]$ gives the smallest knot index $j$ with $x_j$'s prefix $\geq b$,
  • $T[b+1]$ is likewise for $b+1$,
  • localizing $x$ to $T[b]$‥$T[b+1]$ for subsequent interpolation.

2. One-Pass Construction Algorithm

RadixSpline construction proceeds in a single scan utilizing a Greedy Spline Corridor approach (see I. Apostolico et al.), maintaining error bounds on-the-fly and emitting knots as necessary. The essential steps are:

• Initialize the spline with the first key/position.
• For each subsequent distinct key:
  • Update the valid slope corridor $[min\_slope, max\_slope]$ constrained by $E$.
  • If the corridor is violated, emit the previous key as a new knot.
  • After emitting a knot, update the radix table $T$ for all newly covered prefixes.
• After processing all keys, emit the final knot and fill any remaining table entries via left-propagation.
• The resulting spline and radix table are output together.

Construction has $O(N)$ time complexity—each point is processed once, and both corridor updates and knot emissions are $O(1)$. Space usage is $O(M + 2^r)$, where $M \approx N/E$ under uniform keys.

3. Lookup Procedure and Complexity

Lookup for a query key $x$ encompasses three phases:

1. Radix localization: Extract prefix $b = \text{top}_r\_\text{bits}(x)$; obtain bracket indices $j_0 = T[b]$, $j_1 = T[b+1]$ in knots.
2. Knot segment search: Binary search over $knot\_keys[j_0\,..\,j_1-1]$ to find the correct segment for $x$; perform linear interpolation for a position estimate $\hat{p}$.
3. Final key search: Binary search the original array in $$[max(0, \lfloor \hat{p}\rfloor-E),\,min(N{-}1,\lceil\hat{p}\rceil+E)]$$ for the true lower bound.

The key complexity results are:

• $O(1)$ for radix extraction and table lookup.
• $O(\log(M/2^r))$ on average for segment search ($j_1-j_0 \approx M/2^r$).
• $O(\log E)$ for the final binary search in the error window.
• Worst-case is $O(\log N)$, with average-case $O(\log(M/2^r) + \log E)$.

4. Parameter Tuning and Trade-Offs

RadixSpline exposes exactly two parameters:

Parameter	Effect	Trade-off
$E$ (error bound)	Spline fit tightness	Decreasing $E$ increases $M$, trading memory for reduced refinement cost.
$r$ (radix bits)	Radix table granularity	Increasing $r$ grows table size and reduces average knot search interval.

Typical tuning uses $E$ so that $N/E$ fits the memory budget for spline points; $r$ is then set near $\lceil \log_2(N/E) \rceil$, targeting a small knot range per radix slot.

Example for the "face" dataset ($N=200$M):

• $(E=2, r=25)$: $M \approx 100$M, index $\approx 650$MiB, minimal latency.
• $(E=16, r=20)$: $M \approx 12.5$M, index $\approx 200$MiB, latency 11.5% slower—significantly better space/time trade-off.

As $E \rightarrow 0$, $M \rightarrow N$ (degenerating to a full index). As $E$ increases, both build time and memory drop, while latency increases modestly.

5. Empirical Evaluation with SOSD Benchmark

RadixSpline has been extensively evaluated in the SOSD setting (AWS c5.4xlarge, 200M 64-bit keys):

Build Time

Index	Build Time (sec)
BS	NA
B+-tree/ART	0.7
RMI	3–6
RS	0.9

Lookup Latency

Index	Lookup (ns/op)
BS	$\sim$850
B+-tree	$\sim$600
ART	$\sim$300
RMI	120–250
RS	130–280

Index Size

• RS is typically $\sim$100 MiB on most datasets, but can reach 650 MiB with fine-tuning ($E=2$, $r=25$ on "face").
• For $E$ increasing from $2 \to 128$, RS memory falls from 650 MiB$\to$100 MiB and latency increases from $180\,ns \to 300\,ns$.

LSM-Tree Integration (RocksDB, "osmc" dataset)

Replacing per-SSTable B+-tree by RS ($E=16$, $r=22$) for 400M mixed reads/writes:

• Read latency $\downarrow 20\%$
• Write latency $\uparrow 4\%$ (single-pass build during compaction)
• Total execution time $\downarrow 27\%$
• Index memory $\downarrow 45\%$ (allowing larger Bloom filters)

6. Implementation and Optimization Considerations

• RadixSpline is implemented in approximately 100 lines of C++ with no dependencies on external ML libraries.
• Subtracting out any fixed key prefix before radix splitting reduces $r$ and the radix table size.
• Store knot key and position arrays contiguously for cache locality.
• $T[]$ can use 32-bit integers (as $M \leq N \leq 2^{32}$).
• The radix table $T$ is filled incrementally, left-to-right, as knots are emitted, maximizing memory locality.
• In domains with highly skewed key distributions causing some radix slots to cover many knots, a small auxiliary tree per slot is suggested as a fallback.
• The only essential auxiliary operation is a fast integer binary search.

7. Context and Relationships

RadixSpline directly addresses previously noted shortcomings in learned index construction: multi-pass training requirements and implementation complexity. Compared to recursive model index (RMI) approaches—requiring multiple passes with further learned models—RadixSpline achieves competitive lookup and size metrics with a single pass and no ML library dependencies (Kipf et al., 2020). The theoretical underpinnings rely on the error-bounded Greedy Spline Corridor fitting method (I. Apostolico et al.), ensuring formal guarantees on absolute error and supporting efficient online knot placement.

RadixSpline's empirical and theoretical properties make it suitable for a wide variety of workloads, from raw array indexing to integration in LSM-tree storage engines such as RocksDB, where it delivers quantifiable improvements in both speed and memory efficiency.