Query-Preserved Mini-Batching

Updated 4 July 2026

Query-Preserved Mini-Batching is a design principle that constructs mini-batches to maintain key properties—such as graph structure, latent distributions, or gradient freshness—critical for query-dependent tasks.
It spans diverse applications, from graph neural network training with controlled community mixing to latent generative models and SGD optimization, yielding notable speedups and accuracy trade-offs.
Techniques like COMM-RAND, moving-window batches, and minibatch persistency preserve distributional or structural information, ensuring robust gradient estimation while enhancing hardware efficiency.

“Query-Preserved Mini-Batching” is best treated as an Editor’s term: a family of mini-batching schemes in which the batch is constructed so that the property needed by the operative query is preserved under batching, rather than destroyed by purely random sampling or by purely systems-driven grouping. In the supplied literature, the preserved object varies by domain: community-local neighborhoods and label diversity in graph neural network training, set-level latent distribution comparisons in generative models, batch identity across consecutive SGD steps, freshness of gradients in asynchronous optimization, and query-conditioned hard negatives in extreme classification and retrieval (Balaji et al., 25 Apr 2025, Spurek et al., 2019, Fischetti et al., 2018, Attia et al., 2024, Dahiya et al., 2022).

1. Conceptual scope and preserved quantities

Across these works, mini-batching is not a single technique but a constraint on how stochastic computation is organized. In graph learning, the batch must preserve structural locality strongly enough to improve cache behavior, yet retain enough randomness for SGD to converge. In latent-variable generative training, the batch must preserve a set-level distributional comparison even when only one example is processed through the network at a time. In optimization, preservation may mean that the same batch is intentionally reused for $K$ consecutive iterations, or that an asynchronous mini-batch contains only gradients evaluated at the current query point. In extreme classification, preservation refers to maintaining the hard-negative structure conditioned on a query, rather than relying on random in-batch negatives (Balaji et al., 25 Apr 2025, Spurek et al., 2019, Fischetti et al., 2018, Attia et al., 2024, Dahiya et al., 2022).

A concise taxonomy follows.

Setting	Preserved property	Mechanism
GNN training	Community membership coherence, local neighborhood structure, label diversity	Biased root partitioning and biased neighbor sampling
Latent generative training	Empirical latent distribution of size $n$	Moving window of $n-k$ historical latents plus $k$ current examples
SGD / async optimization	Batch identity or gradient freshness at the current query point	Minibatch persistency; stale-gradient filtering
Extreme classification / retrieval	Hard negatives conditioned on a query	Clustered mini-batches and in-batch negatives from neighboring queries

This suggests that the phrase does not denote a single algorithmic lineage. It denotes a design principle: preserve the statistical or structural object that the optimization or retrieval query actually depends on, and then tune the remaining degrees of freedom for hardware efficiency, memory usage, or convergence.

2. Structure-preserving batches in graph neural networks

The clearest explicit formulation appears in "Efficient GNN Training Through Structure-Aware Randomized Mini-Batching" (Balaji et al., 25 Apr 2025). There, mini-batch GNN training has two steps: root node partitioning and neighborhood sampling / subgraph construction. Standard GraphSAGE-style training uses uniform, structure-agnostic randomness in both steps, which improves convergence but produces batches whose nodes are scattered over the graph. The opposite extreme is purely community-based mini-batching: do no shuffling of root nodes across epochs and sample neighbors only within the same community. That yields excellent locality but harms convergence and sometimes accuracy.

COMM-RAND interpolates between those extremes through two explicit knobs. The first is a biased root partitioning knob, expressed as COMM-RAND-MIX- $k\%$ , where training nodes are grouped by community, communities are mixed into super-blocks, and randomization is performed within those super-blocks before batches are cut. The second is a biased neighbor sampling knob $p \in [0.5,1.0]$ , where intra-community edges are given unnormalized sampling weight $p$ and inter-community edges $1-p$, so that

$\Pr(\text{select edge } (v,u)) \propto \begin{cases} p & \text{if } c(u)=c(v)\ 1-p & \text{if } c(u)\neq c(v). \end{cases}$

At $p=0.5$ , there is no community bias; at $n$ 0, only intra-community neighbors are sampled (Balaji et al., 25 Apr 2025).

The preserved object is explicitly twofold. First, the scheme preserves local neighborhood and community membership coherence: root nodes in a batch are predominantly from one or a few communities, and sampled neighborhoods are dominated by intra-community edges. Second, it preserves a distributional property of the batched queries—label diversity—by controlling how many communities are mixed into a batch. The paper reports that higher average label diversity per batch correlates with fewer epochs to convergence, so moderate community mixing is used to avoid the severe slowdown of fully community-based batching (Balaji et al., 25 Apr 2025).

The reported systems effect is substantial. Across reddit, igb-small, ogbn-products, and ogbn-papers100M, COMM-RAND yields up to 2.76× total training speedup, 1.8× speedup on average, with accuracy within 1.79 percentage points of baseline and 0.42 percentage points on average. The best reported setting is COMM-RAND-MIX-12.5% + $n$ 1, with average total training speedup of about 1.8× and a speedup range of 1.14× – 2.76×. The mechanism is reduced per-batch subgraph size and reduced feature working-set size, which improves cache locality; on ogbn-papers100M with a 4M-node software cache, baseline miss rate is 35.46%, while more biased COMM-RAND configurations reduce it to 6.21–6.22% (Balaji et al., 25 Apr 2025).

3. Distribution-preserving latent batches

In "One-element Batch Training by Moving Window" (Spurek et al., 2019), the preserved object is not graph structure but a permutation-invariant set-level query in latent space. The paper considers models with batch objective

$n$ 2

where $n$ 3 is permutation-invariant in its latent arguments. This covers WAE / WAE-MMD, CWAE, SWAE, and more generally set-based or permutation-invariant latent losses. Naively reducing training batch size to 1 makes the latent “sample set” collapse to a single point, so distances such as MMD or Wasserstein become noisy, biased, or degenerate.

The moving-window method decouples the number of examples processed through the network from the effective batch size used to estimate the latent distribution. A theoretical batch size $n$ 4 is fixed, a small current batch size $n$ 5 is chosen, and a moving FIFO buffer of $n$ 6 historical latent codes is maintained. At each step, the distributional loss is computed on the combined latent batch consisting of $n$ 7 historical codes and $n$ 8 current codes, but gradients are backpropagated only through the current examples. The effective empirical latent distribution is therefore

$n$ 9

The paper’s theorem states that if

$n-k$ 0

and $n-k$ 1 solves

$n-k$ 2

then the discrete path $n-k$ 3 generated by the moving-window procedure satisfies

$n-k$ 4

uniformly on compact time intervals (Spurek et al., 2019).

In this setting, query preservation means preserving the multi-sample nature of the latent-space question even when the input-side batch has size 1. The paper compares $n-k$ 5, $n-k$ 6, and $n-k$ 7. On CIFAR-10 for WAE, the reported FID values are 189.71 for $n-k$ 8, 156.14 for $n-k$ 9, and 200.68 for $k$ 0. For CIFAR-10 CWAE, $k$ 1 gives the best FID, 133.82 versus 195.16 for $k$ 2. The method also enables direct CWAE training on FFHQ (1024×1024×3) with $k$ 3, which the paper describes as infeasible with classical $k$ 4 mini-batches because of GPU memory limits (Spurek et al., 2019).

4. Preservation across optimization queries

A different usage appears in "Faster SGD training by minibatch persistency" (Fischetti et al., 2018). Here, preservation means reusing the same mini-batch for $k$ 5 consecutive SGD iterations. Standard SGD corresponds to $k$ 6. With persistency $k$ 7, a fixed batch $k$ 8 is sampled once and then reused for $k$ 9 consecutive iterations, recomputing gradients at the updated parameters each time: $k\%$ 0 The paper studies $k\%$ 1 and also a preliminary adaptive learning-rate variant $k\%$ 2. Within one epoch, each training example is therefore processed not once but $k\%$ 3 times in a row (Fischetti et al., 2018).

The argument is explicitly empirical rather than theoretical: a large mini-batch may be representative enough that one can afford to “slightly overfitting” it without worsening generalization too much, while also reducing data-loading overhead. On CIFAR-10 with a reduced AlexNet, the paper reports that for batch size $k\%$ 4, $k\%$ 5 reaches best accuracy 0.6445 by epoch 30, whereas $k\%$ 6 reaches best accuracy 0.6364 only around epoch 80; yet in wall-clock terms, 100 epochs take about 700 seconds for $k\%$ 7 and about 2000 seconds for $k\%$ 8. For $k\%$ 9, the authors state that “the run with $p \in [0.5,1.0]$ 0 is the clear winner, both in terms of accuracy and loss” (Fischetti et al., 2018).

In "Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batching" (Attia et al., 2024), preservation has yet another form: every gradient in the accepted mini-batch must correspond to the current query point $p \in [0.5,1.0]$ 1. In the asynchronous model, round $p \in [0.5,1.0]$ 2 receives a delayed gradient $p \in [0.5,1.0]$ 3. Algorithm 1 holds the current query point $p \in [0.5,1.0]$ 4 fixed, accumulates a mini-batch, and accepts a gradient only if

$p \in [0.5,1.0]$ 5

which ensures $p \in [0.5,1.0]$ 6. Gradients whose evaluation predates the current inner query are discarded, so the mini-batch is unbiased for the current query point. With quantile-delay notation, the paper derives non-convex rates of the form

$p \in [0.5,1.0]$ 7

and convex smooth rates of the form

$p \in [0.5,1.0]$ 8

and further gives an adaptive method achieving

$p \in [0.5,1.0]$ 9

for non-convex problems and

$p$ 0

for convex smooth problems (Attia et al., 2024).

Taken together, these two lines show that “preservation” may refer either to deliberately reusing a batch across several optimization queries or to strictly filtering so that every element of a mini-batch answers the same optimization query.

5. Query-conditioned negatives and preserved gradient diversity

In "NGAME: Negative Mining-aware Mini-batching for Extreme Classification" (Dahiya et al., 2022), the preserved object is the hard-negative structure conditioned on a query. The setting is deep extreme multi-label classification with an encoder $p$ 1 and a ranking-style triplet objective. NGAME periodically clusters data-point embeddings using balanced hierarchical k-means, samples a mini-batch by drawing clusters, and defines the candidate label set as the union of positives of the queries in that batch: $p$ 2 For each query $p$ 3, in-batch hard negatives are then

$p$ 4

The paper defines $p$ 5-good embeddings and $p$ 6-good clustering, and proves that if the encoder is $p$ 7-good and the clustering is $p$ 8-good, then the missed-hard-negative rate is bounded by

$p$ 9

In the balanced special case, $1-p$0, so the bound becomes $1-p$1 (Dahiya et al., 2022).

This is an explicit query-preservation guarantee: if the embedding preserves positive neighborhoods and the clustering preserves local neighborhoods, then using positives of neighboring queries as in-batch negatives closely approximates the ideal hard-negative set for each query. The paper reports that NGAME allows training with larger mini-batches, is up to 16% more accurate than state-of-the-art methods on benchmark datasets for extreme classification, is 3% more accurate at retrieving search engine queries in response to a user webpage visit to show personalized ads, and in live A/B tests yields up to 23% gains in click-through-rates (Dahiya et al., 2022).

A related but distinct preservation mechanism appears in "Grad Queue : A probabilistic framework to reinforce sparse gradients" (Hasib, 2024). There, the preserved object is not a query-conditioned label neighborhood but rare, informative gradient components that would otherwise be averaged away in large batches. The method maintains a finite queue of online gradients of effective length $1-p$2, typically 3–5, estimates recent mean and standard deviation, and applies a scarcity operator $1-p$3 that amplifies gradients farther than one standard deviation from recent history and attenuates monotonous ones. To minimize conflicting components within large mini-batches, samples are grouped by clustering in inherent feature space, scarcity is measured for each centroid, and centroid gradients are weighted accordingly. The abstract states that the contribution is to restore intra-mini-batch diversity while widening the optimal batch boundary, and reports superior performance for CIFAR10, MNIST, and Reuters News category dataset compared to mini-batch gradient descent (Hasib, 2024).

6. Systems-level batching and broader implications

At the systems level, the same preservation principle appears in stream processing and online ensembles, although the preserved object is now the logical query result or the per-stream update semantics. "Scheduling of Intermittent Query Processing" (Chandrasekaran et al., 2023) studies recurring analytical queries over streams where the result is needed only at a deadline. Instead of tuple-at-a-time or fixed micro-batches, the scheduler processes tuples at selected times in larger batches while preserving the final query result. The cost model is

$1-p$4

and the available slack after a window ends is

$1-p$5

A Custom Query Scheduler implemented on top of Apache Spark chooses larger batches when slack permits, and the performance study with TPC-H data reports orders of magnitude improvement as compared to naively using Spark streaming (Chandrasekaran et al., 2023).

"Improving the performance of bagging ensembles for data streams through mini-batching" (Cassales et al., 2021) applies preservation to online ensemble learning. Here mini-batching does not change the stream’s logical order; it changes the order in which learners and instances are traversed so that each learner is reused for a mini-batch of instances before being evicted from cache. The paper formalizes this with reuse distance. Without mini-batching, reuse distance is $1-p$6; with mini-batch size $1-p$7, it becomes $1-p$8, so mini-batching reduces reuse distance by a constant factor $1-p$9. On six ensemble algorithms and four benchmark datasets, the paper reports speedups of up to 5X on 8-core processors, with a small reduction in predictive performance. It further recommends mini-batches of 50–100 instances as a compromise between speedup and predictive performance, and states that 50 is a conservative starting point when data characteristics are unknown (Cassales et al., 2021).

A plausible implication is that query-preserved mini-batching is best understood as a constrained co-design problem. The preserved quantity is domain-specific—community structure, latent empirical distribution, batch identity, freshness under delay, query-conditioned negatives, rare gradients, or logical query windows—but the pattern is recurrent. The batch is made large enough to exploit hardware locality, amortize overhead, or reduce variance, while explicit mechanisms are added so that the property needed by the query is still present after batching. When those mechanisms are removed, the literature repeatedly reports one of two failure modes: systems-efficient but statistically degraded deterministic batching, or statistically robust but locality-poor random batching.