Index-Batching: Optimizing Bulk Operations

• Index-batching is a technique that groups multiple indexing operations to amortize fixed overheads, yielding improved memory access and execution efficiency.
• It is applied in diverse settings such as PBWT-based genomic matching, GPU-native approximate nearest neighbor search, and batched insertions in B-trees.
• Key trade-offs include processing delay, the need for adapted data layouts and split rules, and balancing throughput against structural quality.

Searching arXiv for the specified papers and closely related work on batching/indexing. Index-batching, as used across several recent arXiv papers, denotes a family of techniques in which multiple index-related operations are collected and processed together so that fixed costs are amortized, memory access becomes more regular, or linear sweeps over compressed structures become feasible. In the cited literature, this idea appears in at least four technically distinct settings: batched reporting from the positional Burrows–Wheeler Transform (PBWT) for haplotype matching, append-only batched updates for GPU-native hybrid approximate nearest-neighbor indices, batched insertions into B-trees under fragmentation analysis, and batched organization of many small sparse systems in accelerator-resident solvers (Gagie, 15 May 2026, Zhao et al., 31 Mar 2026, Bender et al., 12 Mar 2026, Adams et al., 2022). Across these settings, batching is not a single algorithm but a design pattern that couples workload aggregation with data-layout and update-policy choices.

1. Core abstraction and recurring motivation

A common abstraction is that a large number of individually modest operations are expensive when executed one at a time because each operation incurs auxiliary overheads: random-access structures in compressed text or genomic indices, per-kernel launch overheads on GPUs, irreversible split policies in dynamic trees, or low-occupancy execution of many small linear systems. Batching changes the granularity of execution so that the implementation can operate on a group of objects with a single sweep, a single launch, or a single analytical state transition (Gagie, 15 May 2026, Zhao et al., 31 Mar 2026, Adams et al., 2022).

In the PBWT setting, the two phases of set-maximal exact match (SMEM) finding separate discovery from reporting. Phase 1 finds maximal substrings of a query haplotype that occur in the same column-range positions in one or more rows of a haplotype panel, and Phase 2 reports the identities of all haplotypes in which each substring occurs. The cited batching result targets the second phase, which becomes the bottleneck when many haplotypes match each substring or when per-interval access requires extra overhead (Gagie, 15 May 2026).

In GRAB-ANNS, batching is motivated by GPU architecture. The paper argues that directly porting CPU-centric hybrid search algorithms to GPUs leads to severe performance degradation because of irregular memory access, branch divergence, and excessive CPU–GPU synchronization. The index is therefore redesigned around bucketed memory layout, fixed-degree adjacency, and append-only batched insertion (Zhao et al., 31 Mar 2026).

In B-trees, batching has a different role. A batch is a run of consecutive keys inserted at a single position, and the analysis asks how such workloads affect internal fragmentation and long-run space utilization. Here batching is not merely an implementation optimization; it changes the stochastic process analyzed by the generalized Yao framework (Bender et al., 12 Mar 2026).

In the implicit Landau operator, batching gathers all small sparse systems of the same size into a single device-resident kernel launch. The independent systems arise from solving one collision problem per spatial point and species, with roughly uniform dimensions per velocity grid (Adams et al., 2022).

2. PBWT prefix-array access via batching

Haplotype panels are modeled as large binary matrices $M[0..h-1][0..\ell-1]$ of height $h$ and length $\ell$, with total size $n = h\ell$. The PBWT is commonly used to store such panels compactly so that, given a query haplotype $Q$, one can find all SMEMs between $Q$ and the rows of $M$. For each SMEM $Q[i..j]$, Phase 2 reports the interval $a_{j+1}[L..R]$ in the $(j+1)$-th prefix array, whose length is $h$0 (Gagie, 15 May 2026).

Bonizzoni, Gagie, and Gao (2026), as summarized in the batching report, gave two Phase 2 tradeoffs. Scheme A uses $h$1 bits and reports an interval of length $h$2 in $h$3 time. Scheme B uses $h$4 bits and reports in $h$5 time. Here $h$6 is the total number of runs in the run-length encoding of all PBWT columns. The batching result observes that, if reporting can be delayed, the second step can instead use $h$7 bits and constant time per reported haplotype under explicit workload assumptions (Gagie, 15 May 2026).

The key batch size is

$h$8

The method waits until $h$9 SMEM-substrings have been discovered across potentially many queries. If $\ell$0 denotes the total number of haplotypes to report over those $\ell$1 substrings, and if the average satisfies

$\ell$2

then $\ell$3. Under these conditions, all $\ell$4 hits can be reported in $\ell$5 time while using only $\ell$6 bits of additional space (Gagie, 15 May 2026).

The batched reporting algorithm stores one 4-tuple per SMEM, $\ell$7, and sorts the records by increasing end-column $\ell$8. It initializes a doubly-linked list with nodes $\ell$9 in order $n = h\ell$0, together with an array mapping each value to its node. As the prefix arrays are built column by column using linear-time linked-list updates, run boundaries are processed by cutting the list, the pieces are stably reordered by bit value, and the pieces are re-spliced to form $n = h\ell$1. Whenever the current column matches a stored record’s end-column, the algorithm traverses the linked list from the node holding $n = h\ell$2 forward $n = h\ell$3 steps, outputting row IDs (Gagie, 15 May 2026).

The complexity statement is explicit. Building prefix arrays via linked-list updates over $n = h\ell$4 columns costs $n = h\ell$5 total time because the per-column work sums over the number of runs. Traversing all requested intervals costs $n = h\ell$6. Hence the overall time is $n = h\ell$7, which becomes $n = h\ell$8 when $n = h\ell$9, yielding $Q$0 amortized time per reported haplotype. The total space remains $Q$1 bits, including the run-length–compressed PBWT, per-run last-entry annotations, and the batch records (Gagie, 15 May 2026).

The stated assumptions are equally important. The report assumes $Q$2; if $Q$3, it recommends falling back to Scheme B or Scheme A. No randomness is used. A common misconception is that batching in compressed indexing only improves constants; this example shows an asymptotic removal of per-output logarithmic factors, but only after introducing delay and a density assumption on the batched workload. The paper also states that the idea may generalize to RLBWT-based FM-indices, compressed suffix arrays with run-block partitions, and wavelet-tree–based structures when many queries are issued together. This suggests that the decisive property is not PBWT-specific semantics but the availability of a linear sweep that can emit blocks of consecutive identifiers (Gagie, 15 May 2026).

3. GPU-native index-batching in hybrid search

GRAB-ANNS presents index-batching as a GPU-native update pipeline for dynamic hybrid search. Its global data structures are explicitly pre-allocated in large device arrays: a Global Feature Matrix $Q$4 of size $Q$5, a Global Adjacency Matrix $Q$6 of size $Q$7, an Index-to-Bucket Map $Q$8, and Bucket-to-Index Lists $Q$9. Because these structures are pre-allocated, inserting new nodes never requires shifting or re-sorting existing data, and pointers embedded in $Q$0 remain valid for the duration of the process (Zhao et al., 31 Mar 2026).

When a batch $Q$1 of $Q$2 new objects arrives, insertion proceeds in two lock-free, SIMT-friendly operations entirely on the GPU. First, a single 32-bit atomic counter reserves a contiguous block of physical indices $Q$3. In a single kernel, threads write each vector into $Q$4, initialize $Q$5 to empty, and set $Q$6. Second, each thread appends its new physical index to the corresponding $Q$7 list using atomic push operations. The paper states that this two-step append takes $Q$8 time per new vector and preserves all existing graph connectivity (Zhao et al., 31 Mar 2026).

Graph-topology updates are then batched across all $Q$9 insertions in three fused stages rather than processed sequentially. Stage 1 performs parallel candidate search: local search within the bucket via $M$0, global search for remote linking using the graph $M$1, and candidate merging with pruning. Stage 2 performs forward edge selection by heuristic pruning with freshness bias $M$2, overwriting adjacency slots in $M$3 and emitting reverse-link requests. Stage 3 applies reverse rewiring in parallel by attempting to insert the new node into the adjacency list of each affected existing node via small-world pruning (Zhao et al., 31 Mar 2026).

The paper ties batching directly to hardware behavior. Processing all $M$4 insertions in a single launch per stage means the launch overhead is paid only once, while every streaming multiprocessor can stay busy on thousands of threads. The fixed-degree hybrid design is also part of the batching argument: disjoint buckets reduce contention for local updates, the regular $M$5 layout yields coalesced 32- or 64-byte loads and stores, and equal row lengths eliminate intra-warp divergence and dynamic pointer chasing (Zhao et al., 31 Mar 2026).

The complexity description is separated by stage. Physical append is $M$6 per new vector. In Stage 1, each new node performs $M$7 hops within its bucket, where $M$8, plus $M$9 hops on the global graph, and these operations are executed in parallel across $Q[i..j]$0 threads. Forward selection and reverse rewiring are $Q[i..j]$1 per node, again in parallel. The space overhead is $Q[i..j]$2 floats for $Q[i..j]$3, $Q[i..j]$4 integers for $Q[i..j]$5, and $Q[i..j]$6 indices for each of the two maps (Zhao et al., 31 Mar 2026).

The empirical claims are specific. On four 1 M-vector benchmarks—DEEP-96, SIFT-128, GIST-960, and WIT-2048—bulk ingest of 1 M vectors via the static two-pass build takes 30.2 s to 262.8 s depending on $Q[i..j]$7. Using the batch-insertion pipeline, starting from a base index on 0 vectors and dynamically inserting 1 M vectors, completion takes 32.7 s–226.2 s, described as only an 8.2% overhead in the best case and a 13.9% speedup in the worst. Compared against CPU baselines, this corresponds to 2.8–23.8× faster ingestion. The abstract additionally states up to 240.1 times higher query throughput and 12.6 times faster index construction than state-of-the-art CPU-based systems, and up to 10 times higher throughput compared to optimized GPU-native reimplementations, while maintaining high recall (Zhao et al., 31 Mar 2026).

A frequent misconception is that batching on accelerators is mainly a matter of grouping work items. GRAB-ANNS presents a stronger claim: batching becomes effective only when memory layout, update policy, and graph topology are co-designed for SIMT execution. In that sense, index-batching here is inseparable from bucketization and fixed-degree regularity.

4. Batched insertions and fragmentation in B-trees

In the B-tree setting, batching is formalized as a workload model rather than an implementation pattern. A block has capacity $Q[i..j]$8, with $Q[i..j]$9 assumed odd, and when a block reaches size $a_{j+1}[L..R]$0 it must split irrevocably. Under the even-split rule, $a_{j+1}[L..R]$1 keys are split into $a_{j+1}[L..R]$2 and $a_{j+1}[L..R]$3, with tie-breaking to the right sibling. A batched insertion step chooses a leaf at random proportional to its current size $a_{j+1}[L..R]$4, then inserts $a_{j+1}[L..R]$5 consecutive new keys at that point. An equivalent view is to choose an interval uniformly among the $a_{j+1}[L..R]$6 gaps and insert $a_{j+1}[L..R]$7 new keys into that gap in sorted order (Bender et al., 12 Mar 2026).

The space-utilization metric is

$a_{j+1}[L..R]$8

where $a_{j+1}[L..R]$9 is the number of keys and $(j+1)$0 is the total number of blocks. Yao’s classical model with $(j+1)$1 gives $(j+1)$2 under even splitting. The batched analysis generalizes Yao’s recurrence by replacing the $(j+1)$3 birth–death matrix with a matrix $(j+1)$4 that “wraps around” in steps of $(j+1)$5. The main convergence theorem states that $(j+1)$6 is irreducible-Metzler on the subspace of attainable sizes, has a unique positive simple eigenvalue $(j+1)$7, and all other eigenvalues satisfy $(j+1)$8. Consequently, $(j+1)$9, where $h$00 is the unique positive eigenvector satisfying $h$01 (Bender et al., 12 Mar 2026).

The resulting fill bound for even splitting is stated as

$h$02

with $h$03, more precisely $h$04 up to constant factors. The proof sketch describes a step-$h$05 recurrence

$h$06

where $h$07, together with residue-class telescoping products (Bender et al., 12 Mar 2026).

Batching is not uniformly beneficial in this model. The paper explicitly states that even splitting can degrade to $h$08 at $h$09. It therefore develops two remedies. For medium $h$10, an uneven split with a factor $h$11 can confine blocks to at most three sizes and yields proven fill $h$12 for $h$13 and $h$14 for $h$15. For large $h$16, deferred even-split rebalances an overflow into $h$17 nearly equal blocks and yields fill at least $h$18. For $h$19, the rescaled chain becomes exactly Yao’s $h$20-capacity model, giving the exact formula

$h$21

Practical guidance in the exposition includes thresholds such as using standard even split when $h$22, switching to $h$23-uneven splitting for $h$24, and using deferred even splitting when $h$25 to guarantee $h$26 (Bender et al., 12 Mar 2026).

This setting clarifies a central limitation of index-batching: aggregating updates can worsen structural quality if the split rule is not adapted to batch size. The paper therefore treats batching as a workload class that must be matched by a compatible rebalancing policy, not as a universally harmless throughput optimization.

5. Batched problem indexing in implicit Landau solvers

The Landau collision-operator paper uses batching for many small sparse systems that are indexed by spatial point and species. After finite-element discretization and fully implicit time stepping, each velocity grid produces independent systems of the form

$h$27

and in a multi-species problem with $h$28 species and $h$29 spatial points, there are $h$30 independent systems of roughly the same size, about $h$31–$h$32 unknowns (Adams et al., 2022).

All small sparse systems of the same size are gathered into a single kernel launch. For each distinct velocity-space grid, the paper builds batched arrays

$h$33

together with right-hand sides $h$34 and solutions $h$35. The first dimension is the batch, and local row $h$36 of problem $h$37 corresponds to global index $h$38. Padding rounds $h$39 to a multiple of a warp or wavefront and also pads $h$40 for efficient team-parallel loops (Adams et al., 2022).

The TFQMR implementation uses one Kokkos team per system, with vector lanes traversing rows and inner loops. The entire iterative solve for one small system is contained in a single device kernel, and each system retains its own convergence check so that some systems may exit early while others continue. Different grids are not mixed in a single batch; instead, the code launches one kernel per grid, because all systems on one grid share the same matrix size and sparsity pattern (Adams et al., 2022).

The reported performance is architecture-specific. On one A100 node with 4 GPUs, the 2V case with batch size about $h$41 yields Jacobian $h$42 s, mass $h$43 s, and batched-solve $h$44 s per time advance, total $h$45 s. The 3V case with batch size about $h$46 yields Jacobian $h$47 s, mass $h$48 s, and batched-solve $h$49 s, total $h$50 s. On one MI250X node with 8 GCDs, the 2V case with batch size about $h$51 yields Jacobian $h$52 s, mass $h$53 s, and solve $h$54 s, total $h$55 s, while the 3V case with batch size about $h$56 yields Jacobian $h$57 s, mass $h$58 s, and solve $h$59 s, total $h$60 s. The paper also states that batched TFQMR is $h$61–$h$62 faster than an aggregated solver that uses one big block diagonal matrix and standard sparse-BLAS launches (Adams et al., 2022).

Although this is not an indexing data structure in the database sense, it illustrates an important extension of index-batching: the batch index $h$63 becomes the organizing axis for memory layout, scheduling, and hardware occupancy. A plausible implication is that the same principle underlies several ostensibly different systems papers: the useful “index” is often the batch coordinate itself.

6. Cross-cutting trade-offs and scope conditions

Several common trade-offs recur. In PBWT, batching trades delay—waiting for $h$64 SMEMs—for $h$65 amortized time per reported haplotype at $h$66 bits. In GRAB-ANNS, batching trades fine-grained per-node insertion for append-only out-of-order insertion and three bulk graph-update stages, thereby reducing kernel-launch overhead and CPU–GPU synchronization. In B-trees, batching changes the insertion workload and can either preserve or damage utilization depending on the split rule. In the Landau solver, batching requires grouping systems by grid so that matrix sizes and sparsity patterns are homogeneous (Gagie, 15 May 2026, Zhao et al., 31 Mar 2026, Bender et al., 12 Mar 2026, Adams et al., 2022).

Several scope conditions are explicit. PBWT batching assumes $h$67 and falls back when $h$68. GRAB-ANNS relies on pre-allocated dense arrays, fixed-length neighbor lists, and bucket contiguity. The Landau solver emphasizes batching same-size systems and avoiding mixed sizes in one launch. B-tree analysis depends on the ratio $h$69 and shows that even splitting alone is inadequate for some batch regimes (Gagie, 15 May 2026, Zhao et al., 31 Mar 2026, Bender et al., 12 Mar 2026, Adams et al., 2022).

Two misconceptions are directly countered by the cited results. First, batching does not automatically improve all quality metrics: B-tree even splitting can approach the pathological $h$70 bound for $h$71 unless the split strategy is changed (Bender et al., 12 Mar 2026). Second, batching is not merely an implementation detail: in PBWT it removes per-output logarithmic factors under stated assumptions, and in GRAB-ANNS it is inseparable from the index topology and memory layout (Gagie, 15 May 2026, Zhao et al., 31 Mar 2026).

Taken together, these works suggest that index-batching is most effective when three conditions hold simultaneously: there are many independent or weakly coupled operations, the batched objects admit a regular shared representation, and the system can exploit a sweep- or launch-level amortization that is unavailable to isolated operations. That interpretation aligns with the cited domains without collapsing their differences: compressed genomic reporting, GPU graph indexing, B-tree occupancy, and batched sparse solves each instantiate the same systems principle under very different mathematical and architectural constraints.