---
title: 'DiveBatch: Adaptive Batching Methods'
url: https://www.emergentmind.com/topics/divebatch
type: topic
---

# DiveBatch: Adaptive Batching Methods

“DiveBatch” does not denote a single universally standardized algorithm in the arXiv literature. Instead, the name is used, or mapped retrospectively, across several lines of work concerned with batch construction, batch scheduling, or batch-size adaptation. In the DPP-based stochastic optimization literature, DiveBatch corresponds to Diversified Mini-Batch SGD (DM-SGD), where a Determinantal Point Process (DPP) or $k$-DPP samples repulsive mini-batches [1705.00607]. In later work, closely related labels refer to reference-model-free diversified subset selection during training [2406.04872], gradient-diversity-aware adaptation of the mini-batch size itself [2509.16173], dynamic or automatic batching of computation graphs [1705.07860], and diversified batched decision making in Bayesian optimization and offline reinforcement learning [1110.3347]. Taken together, these usages suggest an umbrella notion: batch formation is no longer treated as a passive systems detail, but as an explicit optimization object.

## 1. Naming and conceptual scope

The available usages of “DiveBatch” span several technically distinct problem settings. In some cases the name is explicit; in others it is a descriptive label attached to a method whose published name differs.

| Usage | Core mechanism | Representative source |
|---|---|---|
| Diversified mini-batch SGD | DPP/$k$-DPP sampling over data indices | DM-SGD [1705.00607] |
| Diversified batch selection | Group-wise orthogonalized representativeness | DivBS [2406.04872] |
| Adaptive batch-size SGD | Batch size proportional to estimated gradient diversity | DIVEBATCH [2509.16173] |
| Automatic batching | Runtime grouping of identical operations in dynamic graphs | DyNet auto-batching [1705.07860] |
| Dynamic batch BO | Adaptive parallel query count based on near-independence | Dynamic Batch BO [1110.3347] |
| Diversified batched BO | DPP-reweighted Thompson sampling | DPP-BBO / DPP-TS [2110.11665] |
| Safe diversified batch RL | Diverse policy ensembles under support and safety constraints | Safe Diversified MBPS [2411.09722] |

This multiplicity matters because the common intuition—“improve batching by injecting structure”—masks fundamentally different objects of control. In DM-SGD and DivBS, the batch is a subset of training examples. In DIVEBATCH, the controlled variable is the batch size. In dynamic computation-graph systems, the “batch” is an execution schedule over graph nodes rather than a statistical sample. In Bayesian optimization and offline RL, the batch is a parallel set of decisions or policies. This suggests that “DiveBatch” is best understood functionally rather than nominally: it refers to methods that make batch composition, size, or execution order adaptive and non-uniform, rather than fixed and uniform [1705.00607, 2406.04872, 2509.16173, 1705.07860, 1110.3347, 2110.11665, 2411.09722].

## 2. DPP-based diversified mini-batching

In the terminology of “Determinantal Point Processes for Mini-Batch Diversification,” DiveBatch corresponds to Diversified Mini-Batch SGD (DM-SGD), which replaces uniform mini-batch sampling with DPP-based sampling [1705.00607]. The method starts from a positive semidefinite kernel $L \in \mathbb{R}^{n \times n}$ encoding similarity among the $n$ training examples. For an $L$-ensemble DPP,
$$
P(S) = \frac{\det(L_S)}{\det(L + I)},
$$
and for a fixed-size $k$-DPP,
$$
P(S) = \frac{\det(L_S)}{e_k(L)}, \qquad |S| = k.
$$
Here $L_S$ is the principal submatrix indexed by $S$, and $e_k(L)$ is the $k$-th elementary symmetric polynomial of the eigenvalues of $L$.

The operational idea is repulsion. DPPs give low probability to subsets containing redundant points and higher probability to subsets containing diverse points. In the unconditioned DPP, the marginal kernel is $K = L(L+I)^{-1}$ and the inclusion covariance satisfies
$$
\operatorname{Cov}(1_{i \in S}, 1_{j \in S}) = -K_{ij}^2 \le 0 \quad (i \ne j),
$$
so sampling one index decreases the probability of sampling similar indices. The fixed-size $k$-DPP does not preserve negative dependence for every pair in the same strict form, but the practical behavior remains repulsive for nearby or very similar items. This directly targets the failure mode of classical uniformly sampled SGD on imbalanced or near-duplicate data: redundant samples co-occur, gradients become positively correlated, and the mini-batch estimator becomes unnecessarily noisy.

The gradient analysis in DM-SGD is stated in terms of the empirical objective
$$
f(w) = \frac{1}{n}\sum_{i=1}^n \ell_i(w), \qquad \nabla f(w)=\frac{1}{n}\sum_{i=1}^n \nabla \ell_i(w),
$$
and the mini-batch estimator
$$
\hat g(w)=\frac{1}{|S|}\sum_{i \in S}\nabla \ell_i(w).
$$
With $m_i \in \{0,1\}$ the inclusion indicator, $b_i=\mathbb{E}[m_i]$, and
$$
C_{ij}=\frac{\mathbb{E}[m_i m_j]}{b_i b_j}-1,
$$
the variance decomposition for
$$
g^*=\frac{1}{k}\sum m_i g_i, \qquad g_i=\nabla \ell_i(w),
$$
is
$$
\operatorname{Var}(g^*)=
\frac{1}{k^2}\sum_{i=1}^n (b_i-b_i^2)\|g_i\|_2^2
+
\frac{1}{k^2}\sum_{i \ne j} C_{ij} b_i b_j g_i^\top g_j.
$$
The sufficient condition
$$
C_{ij} g_i^\top g_j < 0 \quad \text{for all } i \ne j
$$
implies variance reduction relative to uniform sampling. The interpretation is explicit: when similar samples produce positively correlated gradients, a repulsive sampler makes $C_{ij}$ negative on those pairs, so the covariance term subtracts variance rather than adding it.

Kernel construction determines what “diversity” means. The paper uses linear kernels $L=BB^\top$, cosine kernels, and RBF kernels. For text, tf-idf vectors with an annealed linear kernel $L(x_i,x_j)=x_i^\rho x_j^\rho$ with $\rho=0.1$ were used; for images, label-feature blends such as $F=[(1-w)X,wH]$ and $L=FF^\top$ allow an explicit trade-off between class balance and intra-class feature diversity. Uniform sampling is recovered when $L$ is proportional to the identity, and biased stratified sampling is recovered from a block-diagonal $L$ with all-ones within strata and zeros across strata. The method therefore generalizes stratified sampling beyond discrete bins.

The optimized objective can be viewed as a diversified risk,
$$
J^*(\theta)=\frac{1}{k}\sum_{i=1}^n b_i \,\ell(x_i;\theta),
$$
which is biased relative to the original empirical risk. If unbiasedness is required, importance weighting yields
$$
\theta_{t+1}=\theta_t-\rho_t \frac{1}{k}\sum_{i \in S}\frac{1}{b_i}\nabla \ell(\theta,x_i),
$$
and
$$
\mathbb{E}\!\left[\frac{1}{k}\sum \frac{m_i}{b_i}g_i\right]
=
\frac{1}{k}\sum g_i.
$$
The paper notes, however, that the biased diversified risk often performs better on imbalanced data.

The reported empirical profile is correspondingly mixed across unsupervised and supervised settings. On Reuters R8, DM-SVI improved average per-class accuracy from $82.11\%$ to $87.24\%$ and total accuracy from $94.11\%$ to $94.70\%$. On Oxford 102 Flowers, the best DM-SGD result at $k=80$ reached $86.7\%$ versus a baseline of $81.8\%$. On balanced MNIST, benefits remained visible through lower variance and improved learning curves, indicating that the method is not restricted to overt class imbalance [1705.00607].

## 3. Reference-model-free diversified batch selection

A second, distinct DiveBatch usage is Diversified Batch Selection (DivBS), described in “Diversified Batch Selection for Training Acceleration” and also referred to there as DiveBatch [2406.04872]. Here the setting is online batch selection: at each training step, one draws a large candidate batch $B$ of size $N_B$ and selects a smaller subset $S \subset B$ of size $N_S < N_B$ for the optimizer step. The paper’s criticism of prior reference-model-free methods is that sample-wise top-$N$ scoring ignores redundancy: multiple high-scoring but near-duplicate samples can be selected simultaneously.

DivBS addresses that failure mode by replacing additive sample-wise scoring with a group-wise orthogonalized representativeness objective. Let $U=g(B,\theta)=\{g(d,\theta): d \in B\}$ be the selection features, taken in the paper to be last-layer per-sample gradients. For a subset $S$, let $(g(S,\theta))$ denote the set of orthonormal bases of the span of $g(S,\theta)$. The representativeness objective is
$$
r(S,B,\theta)=\max_{E \in (g(S,\theta))}\sum_{e \in E}\sum_{u \in g(B,\theta)} e \cdot u.
$$
A key simplification in the paper is
$$
r(S,B,\theta)
=
\sqrt{|E| \sum_{e \in E}\left(e \cdot \sum_{u \in g(B,\theta)}u\right)^2},
\qquad \forall E \in (g(S,\theta)).
$$
Thus the objective measures the Euclidean norm of the projection of the batch-sum vector onto the subspace spanned by the selected examples, scaled by the number of orthogonal directions. Redundancy is explicitly removed by orthogonalization.

The associated auxiliary function
$$
r'(S,B,\theta)=
\sqrt{\sum_{e \in E}\left(e \cdot \sum_{u \in g(B,\theta)}u\right)^2},
\qquad \forall E \in (g(S,\theta)),
$$
is normalized, monotone, and submodular. Consequently, greedy maximization under a cardinality constraint attains the classical $(1-e^{-1})$ approximation guarantee. The theory-friendly greedy rule orthogonalizes every candidate by Gram–Schmidt residualization,
$$
e(d)=
\frac{g(d,\theta)-\sum_{e \in E}(e \cdot g(d,\theta))e}
{\left\|g(d,\theta)-\sum_{e \in E}(e \cdot g(d,\theta))e\right\|},
$$
and selects the candidate maximizing $|e(d)\cdot \mathrm{Sum}|$, where $\mathrm{Sum}=\sum_{u \in g(B,\theta)}u$.

The practical DivBS algorithm avoids per-candidate orthogonalization at each step. It instead selects
$$
d = \arg\max_{d \in B} |g(d,\theta)\cdot \mathrm{Sum}|,
$$
orthogonalizes only the chosen sample, and updates
$$
\mathrm{Sum} \leftarrow \mathrm{Sum} - (e \cdot \mathrm{Sum})e.
$$
This preserves the “explain the remaining batch signal” interpretation while reducing computational cost from the greedy procedure’s candidate-wise residualization to a substantially cheaper realization.

Empirically, DivBS is evaluated on CIFAR-10, CIFAR-100, Tiny ImageNet, CIFAR-100-LT, PASCAL VOC 2012 trainaug, Wikipedia cross-modal retrieval, and GPT-2 Medium with LoRA on the E2E NLG Challenge. At a $30\%$ budget, CIFAR-100 accuracy rises from $75.98\%$ under uniform selection to $77.21\%$ under DivBS, close to the full-data result of $77.28\%$. On CIFAR-10, the corresponding numbers are $94.61\%$ for uniform and $95.07\%$ for DivBS, with a full-data result of $95.50\%$. On PASCAL VOC trainaug, DivBS reaches $69.85$ mIoU at $30\%$ budget versus $68.56$ for uniform. The paper also reports that selection overhead is $\le 5\%$ extra versus uniform sampling. These results place DiveBatch, in this sense, at the intersection of online subset selection, gradient-space coreset construction, and diversity-aware training acceleration [2406.04872].

## 4. Gradient-diversity-aware batch-size adaptation

A third usage is the explicitly named “DIVEBATCH: Accelerating Model Training Through Gradient-Diversity Aware Batch Size Adaptation” [2509.16173]. Unlike DM-SGD or DivBS, this method does not alter which samples are chosen within a candidate set. It dynamically adjusts the mini-batch size itself in proportion to an estimate of gradient diversity.

The central quantity is
$$
\Delta_S(\theta):=
\frac{\sum_{i=1}^n \|\nabla_\theta \ell(\theta;z_i)\|_2^2}
{\left\|\sum_{i=1}^n \nabla_\theta \ell(\theta;z_i)\right\|_2^2}.
$$
If individual gradients are aligned, the denominator is large and $\Delta_S(\theta)$ is small; if gradients are diverse or cancel, the denominator is small and $\Delta_S(\theta)$ is large. DiveBatch estimates this quantity at epoch boundaries by accumulating, over the batches of epoch $k$, the sum of per-sample gradient norm squares and the norm of the accumulated gradient sum:
$$
\hat{\Delta}_S(\theta^{t_{k+1}})=
\frac{\sum_j \sum_{i \in B_j}\|\nabla_\theta \ell(\theta^{t_k+j-1};z_i)\|_2^2}
{\left\|\sum_j \sum_{i \in B_j}\nabla_\theta \ell(\theta^{t_k+j-1};z_i)\right\|_2^2}.
$$
The update rule is then
$$
m_{k+1} \leftarrow \min\{m^{\max}, \delta \cdot n \cdot \hat{\Delta}_S(\theta^{t_{k+1}})\},
$$
with scaling parameter $\delta>0$, small initial batch size $m_0$, and maximum batch size $m^{\max}$.

The theoretical motivation is imported from convergence analyses in which the “safe/effective” mini-batch size scales like $n \cdot \Delta_S$. The paper states that if $m = \delta \cdot n \cdot \Delta_S(\theta^t)+1$, one mini-batch SGD step reduces the expected squared distance to the minimizer approximately $m$ times more than one single-sample step, up to second-order terms controlled by
$$
M^2(\theta):=\frac{1}{n}\sum_{i=1}^n \|\nabla_\theta \ell(\theta;z_i)\|_2^2.
$$
The paper also presents standard convergence bounds under Lipschitz-gradient and bounded-variance assumptions, emphasizing that diversity directly modulates the variance/covariance structure of mini-batch gradients.

The algorithmic cost is substantial because exact implementation requires per-sample gradients. The reported average peak memory on CIFAR-10 is $717$ MB for SGD$(128)$, $9565$ MB for SGD$(2048)$, $6751$ MB for AdaBatch$(128 \rightarrow 2048)$, and $13164$ MB for DiveBatch$(128 \rightarrow 2048)$. The implementation uses Backpack-for-PyTorch for per-sample gradients, and the paper notes that micro-batching or proxy statistics could reduce cost, although the reported experiments use exact per-sample sums.

The empirical picture is explicitly framed as a speed–accuracy trade-off. On CIFAR-10, the time to reach $\pm 1\%$ of final accuracy is $638$ s for DiveBatch, versus $1361$ s for AdaBatch, $3189$ s for SGD$(128)$, and $1557$ s for SGD$(2048)$; final validation accuracies are $93.82$ for DiveBatch, $94.88$ for AdaBatch, $95.50$ for SGD$(128)$, and $92.47$ for SGD$(2048)$. On CIFAR-100, the corresponding times are $1192$ s, $1263$ s, $2808$ s, and $1098$ s, with final accuracies $71.75$, $72.99$, $74.92$, and $67.59$. On Tiny-ImageNet, DiveBatch reaches $51.66$ final accuracy versus $53.72$ for AdaBatch and $55.49$ for SGD$(256)$, but reduces time to near-final accuracy from $9703$ s to $6478$ s. The stated overall range is $1.06$–$5.0\times$ faster convergence than standard SGD and AdaBatch, with a slight trade-off in performance [2509.16173].

## 5. Dynamic and automatic batching of computation graphs

A different literature uses DiveBatch as a descriptive label for automatic or dynamic execution batching in neural computation graphs. In “On-the-fly Operation Batching in Dynamic Computation Graphs,” the problem is not statistical sampling but execution scheduling: automatically group semantically identical operations from different parts of a dynamic graph into one batched kernel, even when architectures involve variable-length sequences, trees, or input-dependent control flow [1705.07860]. DyNet accomplishes this by separating graph construction from execution via lazy evaluation, assigning each node a batching signature that encodes operator type, shapes, parameter identities, broadcasting semantics, and attributes, and then applying an agenda-based scheduler. The scheduler repeatedly selects a ready node, collects all ready nodes with the same signature, and executes them together. Exact optimal batching-aware scheduling is stated to be NP-hard, so the system uses heuristics based on average depth and operator cost class.

This runtime perspective yields large throughput improvements without developer-managed padding or masking. On a synthetic BiLSTM benchmark with fixed length $40$ and batch size $64$, CPU throughput improves from about $193$ ms/sentence to about $16.9$ ms/sentence under the agenda-based strategy, approximately an $11\times$ speedup; GPU throughput improves from about $54.6$ ms/sentence to about $5.03$ ms/sentence, also approximately an $11\times$ speedup. On more irregular tasks such as TreeLSTM and transition-based dependency parsing, substantial gains remain, although smaller for strongly sequential control flow [1705.07860].

“Effective Approaches to Batch Parallelization for Dynamic Neural Network Architectures” presents a related but more manual formulation based on grouping identical module calls across a minibatch of dynamic graphs [1707.02402]. If each example has graph size at most $s$ and module vocabulary size $p$, standard topological batching yields at most $ps$ expensive module calls rather than $bs$ under naive per-example execution. For predetermined trees or DAGs with depth $d$, improved depth-based batching yields at most $pd$ calls, and for a balanced tree this becomes $O(p \log_2 s)$. In sparsely gated mixture-of-experts layers, the approximate speedup is
$$
S \approx \frac{kb}{n},
$$
where $n$ is the number of experts and each example activates $k \ll n$ experts. The paper reports speedups of over $10\times$ on modular program execution and up to $1000\times$ for sparsely gated MoE layers [1707.02402].

ACRoBat pushes this line toward compiler-driven divergence-aware auto-batching [2305.10611]. It combines inline depth computation, program phases, ghost operations that delay premature dispatch across conditionals, grain-size coarsening of static subgraphs, tensor-dependent control-flow support via fibers, parameter-reuse inference, horizontal fusion, gather-operator fusion, and ahead-of-time host code generation. The grouping key is a tuple of operator type, depth, phase, reuse signature, and layout signature. The framework reports up to $8.5\times$ better performance than DyNet on an Nvidia GeForce GPU, with substantial reductions in DFG construction, scheduling overhead, memory copies, and CUDA API time. In this dynamic-systems sense, DiveBatch names a family of execution planners whose objective is to recover batch parallelism from control-flow divergence rather than from data subsampling [2305.10611].

## 6. Diversified batching in Bayesian optimization, offline RL, and distributed training

In Bayesian optimization, DiveBatch refers to selecting multiple parallel evaluations while staying close to a strong sequential acquisition rule. “Dynamic Batch Bayesian Optimization” adapts the batch size $p_t$ at each BO iteration rather than fixing it [1110.3347]. The method is built around Expected Improvement (EI), the Gaussian-process posterior
$$
\mu(x)=k(x,X)^\top K^{-1}y, \qquad
\sigma^2(x)=k(x,x)-k(x,X)^\top K^{-1}k(x,X),
$$
and the observation that a candidate can be added to the current batch if its posterior mean is nearly independent of the as-yet-unobserved outcomes of previously selected points. The decision rule is based on an outcome-independent bound on the expected absolute mean change,
$$
\mathbb{E}_{y^*}[|\Delta^*(\mu_z)|]
\le
\left\|
(PA^{-1}B^\top-k_z^*)m
\right\|_\infty
\sqrt{\frac{2}{\pi}\|\sigma^*\|_1},
$$
and a point is accepted into the current batch if this quantity is at most $\epsilon$. Reported speedups are in the $6$–$18\%$ range with little impact on performance across eight benchmarks [1110.3347].

A distinct BO formulation introduces DPPs directly into batched Thompson sampling. “Diversified Sampling for Batched Bayesian Optimization with Determinantal Point Processes” defines
$$
P_{\mathrm{DPP\text{-}TS},t}(X)
\propto
P_{\max,t}(X)\,\det(I+\sigma^{-2}K_{t,X}),
$$
where $P_{\max,t}(X)=\prod_{x_b \in X} p_{\max,t}(x_b)$ is the Thompson-sampling maximum distribution and the determinant term enforces mutual-information-based diversity [2110.11665]. The MCMC acceptance probability simplifies to
$$
\alpha=\min\left\{1,\frac{\det(L_{t,X'})}{\det(L_{t,X})}\right\}.
$$
The paper proves a Bayesian simple regret bound tighter than that of classical batched TS through an explicit negative $C_3$ term, reflecting a diversity bonus.

In offline reinforcement learning, the directly relevant name is not DiveBatch but iterative batch reinforcement learning via Safe Diversified Model-based Policy Search [2411.09722]. The method trains an ensemble of $K$ policies in a learned differentiable simulator, enforces support and safety via behavior regularization, likelihood-threshold penalties, or direct action constraints, and rewards diversity among trajectories through the minimum lock-step Euclidean distance,
$$
D(T_1,\ldots,T_K)=
\frac{1}{H}\min_{k' \ne k}\sum_{t=1}^H \|s_{k,t}-s_{k',t}\|_2.
$$
A representative objective is
$$
L(\theta)=
-\frac{1}{NKH}\sum \gamma^t e(s_{k,t},a_{k,t})
+
\alpha_s \frac{1}{KH}\sum \max(G(s_{k,t},\pi(s_{k,t};\theta_k))+\delta,0)
-
\alpha_d \frac{1}{H}\min_{k' \ne k}\sum_t \|s_{k,t}-s_{k',t}\|_2.
$$
On the Industrial Benchmark, diversity accelerated cost reduction and lowered variance under both constrained-policy and soft-constraint variants.

A further batch-adaptation use appears in distributed deep learning. DBS dynamically adjusts each worker’s mini-batch size according to measured throughput while keeping the global batch size fixed [2007.11831]. With worker throughputs $\hat r_i^{(t)}$, the proportional allocation rule is
$$
\tilde B_i^{(t+1)} = B \cdot \frac{\hat r_i^{(t)}}{\sum_{k=1}^n \hat r_k^{(t)}},
$$
followed by integer rounding and dataset repartitioning through DynamicDatasetAdjust. The synchronized gradient aggregation remains
$$
g^{(t)}=\sum_{i=1}^n \frac{B_i^{(t)}}{B} g_i^{(t)},
\qquad
w_{t+1}=w_t-\eta g^{(t)},
$$
so the statistical semantics are those of standard mini-batch SGD, while wall-clock efficiency improves on heterogeneous clusters.

## 7. Specialized domain-specific formulations

Some uses of DiveBatch are highly application-specific but retain the same structural theme: a bottleneck batch dimension is redesigned so that scalability is recovered without sacrificing the global coupling that makes the original problem hard.

In numerical PDE solving, the term is used for a batched GPU methodology in which many independent linear systems
$$
A x^{(m)} = b^{(m)}, \qquad m=1,\ldots,M,
$$
share the same banded left-hand side $A$ [2107.05395]. The implementation stores only one global copy of the tridiagonal or pentadiagonal matrix, pre-factorizes it once, and solves all right-hand sides with an interleaved memory layout that yields coalesced accesses. Relative to cuThomasBatch and cuPentBatch, this reduces left-hand-side storage by about $75\%$ for tridiagonal systems and about $83\%$ for pentadiagonal systems, with reported speedups up to about $2\times$–$2.4\times$ for the tridiagonal case and about $1.2\times$–$1.5\times$ for the pentadiagonal case at large problem sizes. The paper demonstrates $2^{20}$ independent one-dimensional Cahn–Hilliard simulations on a single GPU [2107.05395].

In deep subspace clustering, a mini-batch strategy replaces full-batch self-expressive training with a memory bank of latent features [2507.19917]. Classical DSC requires
$$
\hat Z = C Z,
$$
with global coefficient matrix $C \in \mathbb{R}^{N \times N}$ and full latent matrix $Z \in \mathbb{R}^{N \times h}$, which forces full-batch processing. The proposed batch-wise DSC (BDSC) stores a memory bank $M \in \mathbb{R}^{N \times h}$ and uses
$$
\hat Z_B = C_{B,:} M
$$
for the current batch. The AE-based objective becomes
$$
\mathcal{L}^{(B)}=
\|X_B-\hat X_B\|_F^2
+
\alpha \|Z_B-C_{B,:}M\|_F^2
+
\beta \|C\|_2,
$$
while the contrastive variant CLBDSC replaces reconstruction with an InfoNCE term that uses the bank as a source of negatives. On COIL100 with $128 \times 128$ inputs and a ResNet-18 encoder, the paper reports ACC/NMI of $0.813/0.963$ for BDSC and $0.829/0.964$ for CLBDSC; on ORL at $128 \times 128$, the corresponding numbers are $0.935/0.970$ and $0.923/0.955$ [2507.19917].

These specialized uses underscore a broader pattern. Whether the obstacle is the $N \times N$ self-expressive matrix of subspace clustering, the repeated left-hand side of banded linear systems, the control-flow divergence of dynamic neural programs, or the redundancy of uniformly sampled gradients, DiveBatch-style methods intervene at the batch layer itself. The intervention may take the form of repulsive subset selection, orthogonalized representativeness, gradient-diversity-based resizing, execution-graph scheduling, safe diversified policy ensembles, or shared-structure amortization. What remains constant is the rejection of naive fixed batching in favor of batch policies that encode geometry, dependence, safety, or systems structure.

Source: https://www.emergentmind.com/topics/divebatch