Batch Variable-length Indexing (BVI)
- Batch Variable-length Indexing (BVI) is a differentiable operator that selectively extracts and restores masked latent tokens in DiT-based video object removal.
- It replaces dense full-token diffusion with a mask-aware mechanism using complementary forward and backward indices to limit computations to masked regions.
- BVI enables batchwise variable-length token processing, yielding significant training and inference acceleration while maintaining visual quality.
Searching arXiv for the cited papers to ground the article in current records. Batch Variable-length Indexing (BVI) is a differentiable dynamic indexing operator introduced in YOSE, “You Only Select Essential Tokens for Efficient DiT-based Video Object Removal,” to make DiT-based video object removal mask-aware by selecting only the latent tokens corresponding to the masked region, processing those tokens, and restoring them to their original spatial-temporal locations (Wu et al., 30 Apr 2026). Its defining role is to support variable-length token sequences across samples in a batch while preserving end-to-end optimization, thereby replacing dense full-token diffusion with selective computation whose cost scales with the masked area rather than the total spatiotemporal token count. In this formulation, BVI is a within-batch token-selection and restoration mechanism, not a general-purpose indexing architecture.
1. Definition and scope
BVI is defined as the core token-selection mechanism in YOSE. The operator produces two complementary index sets: the forward index IndF, which selects the essential tokens to be processed by the DiT blocks, and the backward index IndB, which restores the processed tokens back to their original spatial-temporal locations (Wu et al., 30 Apr 2026). In YOSE, “essential tokens” are exactly the latent tokens corresponding to masked spatial-temporal blocks.
The motivation is explicitly tied to video object removal. DiT-based methods perform dense computations over the entire spatiotemporal token space even when only a small masked region actually requires processing. The paper reports that around 70% of real cases have mask ratios below 20%, so selective computation is highly beneficial. This establishes BVI as a mask-aware acceleration mechanism rather than a generic sparse-transformer abstraction.
The scope of BVI is correspondingly narrow and technical. It is introduced for efficient fine-tuning and inference in DiT-based video object removal, and the paper’s main system-level claim is that YOSE changes inference from full-token constant cost to mask-ratio-dependent linear cost while maintaining visual quality comparable to the baseline (Wu et al., 30 Apr 2026). A plausible implication is that BVI is best understood as a specialized differentiable gather/scatter layer for masked latent tokens under batchwise variable-length constraints.
2. Formal operator and differentiable indexing
The paper first states the intuitive hard-indexing formulation as
$St_{in} = Videomask[mask]. \tag{1}$
After processing,
$Videomask[mask] = St_{out}. \tag{2}$
It then identifies direct discrete selection such as Videomask[mask] as unsuitable for end-to-end optimization because it is not differentiable in the usual indexing sense (Wu et al., 30 Apr 2026).
BVI replaces hard tensor indexing with differentiable coordinate sampling. Using normalized coordinate grids, token selection is defined as
$St_{in} = GSample(Videomask, Ind_F), \tag{3}$
where GSample refers to a Torch grid_sample-like function. Restoration is defined as
$Video_{out} = GSample(St_{out}, Ind_B). \tag{4}$
This gives BVI the character of a soft, differentiable gather/scatter operator over masked tokens.
The significance of this reformulation is twofold. First, it addresses gradient blocking by avoiding non-differentiable discrete indexing. Second, it provides a unified mechanism for extracting and restoring variable-length token subsets within standard tensor-based training pipelines. The paper therefore frames BVI not merely as an indexing convenience, but as the enabling operator that makes selective token computation trainable.
3. Batchwise variable-length token processing
One of BVI’s central contributions is its explicit support for variable-length token sequences across samples in a batch. Different videos may contain different numbers of masked tokens, so the selected token count differs per sample. Standard transformers assume a fixed token length, which would otherwise force inefficient padding or break batch parallelism (Wu et al., 30 Apr 2026).
The paper gives Algorithm 1: Batch Variable-length Indexing. Its forward phase counts the number of masked tokens in each sample,
C = Length(mask[0]),
then builds a normalized coordinate sample grid,
Sample = Linspace(1/2C - 1, 1 - 1/2C, C).
For each sample i, it extracts the coordinates where mask[i] is true,
This_Ind = Sample[mask[i]],
appends them to IndF, and records the selected-token length as LenF[i]. It then computes
Lmax = Max(LenF)
and pads IndF for shorter samples so that all batch elements share the same effective length. In the backward phase, it constructs a corresponding backward index tmpB for each sample and stores these as IndB (Wu et al., 30 Apr 2026).
The operator therefore preserves batchability by combining per-sample variable-length selection with batch-level alignment to the maximum masked length. The paper emphasizes that computation becomes proportional to the number of masked tokens, not the total number of video tokens. This is the technical sense in which BVI is “batch variable-length”: variable-length processing is handled inside a batched execution model rather than by abandoning batching.
4. Mask alignment in latent space and interaction with DiffSim
In YOSE, BVI does not operate on a naively resized image-space mask. Because the video is encoded by a 3D-VAE with spatiotemporal strides, the paper introduces mask embedding so that the mask is aligned with latent tokenization (Wu et al., 30 Apr 2026). For each VAE block, a latent token is marked masked if any pixel in its receptive block is masked:
$1 - \prod (1 - m_i), \tag{15}$
where is the binary mask value inside the block. Once this latent-space mask is built, BVI selects the masked-region tokens.
BVI is paired with the Diffusion Process Simulator (DiffSim), which addresses the information loss caused by omitting unmasked tokens from direct processing. The division of labor is explicit: BVI selects and processes only inner or masked tokens, while DiffSim simulates the influence of outer or unmasked tokens without explicitly running full-token DiT attention (Wu et al., 30 Apr 2026). DiffSim uses BVI to create IndF_in, IndB_in for inner tokens and IndF_out, IndB_out for outer tokens.
The relevant DiffSim equations begin from the residual
$Res_{Nis} = Lat_{Nis} - Lat_{mask}. \tag{5}$
A learnable combining parameter defines
This is then modulated by learnable scaling and bias,
$KV = (1 + S[i])\cdot KV + Bias[i]. \tag{7}$
Finally, processed inner tokens are mapped back through backward BVI as
$Videomask[mask] = St_{out}. \tag{2}$0
This interaction clarifies BVI’s systems role. BVI supplies the variable-length masked-token pathways required for selective computation; DiffSim compensates for the omitted outer context. A plausible implication is that BVI by itself addresses efficiency, whereas the BVI-plus-DiffSim composition addresses efficiency and semantic consistency simultaneously.
5. Computational behavior, training efficiency, and empirical findings
The paper’s principal efficiency claim is that YOSE achieves mask-aware acceleration, with inference time scaling approximately linearly with the masked regions, in contrast to full-token diffusion methods whose computation remains constant regardless of mask size (Wu et al., 30 Apr 2026). It further states that the FLOPs expression $Videomask[mask] = St_{out}. \tag{2}$1, as a function of mask ratio $Videomask[mask] = St_{out}. \tag{2}$2, is linear in $Videomask[mask] = St_{out}. \tag{2}$3.
Empirically, the reported savings are substantial. The paper reports 3.3× acceleration at 5% mask ratio and 2.5× speedup at 20% mask ratio, which it identifies as common in real video object removal cases. It also states that when the mask becomes very large, runtime converges toward full-token inference, so YOSE does not become slower than the baseline in the worst case. In the abstract, YOSE is reported to achieve up to 2.5X speedup in 70% of cases while maintaining visual quality comparable to the baseline (Wu et al., 30 Apr 2026).
BVI also affects training efficiency. The training setup described for YOSE uses batch size 32 on 8 GPUs, 17 input frames, resolution 480 × 832, AdamW, learning rate $Videomask[mask] = St_{out}. \tag{2}$4, 2K training steps, and about 4 hours total on 70K samples. The paper notes that with BVI, training can be done in about 4 hours, whereas conventional approaches that cannot handle variable-length tokens would take about 11 hours, yielding nearly a 3× training acceleration (Wu et al., 30 Apr 2026).
The ablation discussion assigns BVI a specific functional role: it enables multi-batch training under variable-length masks and greatly improves training efficiency. DiffSim and fusion preserve visual quality. The paper reports similar or better VBench scores and comparable background PSNR/SSIM/LPIPS, and notes that YOSE can improve background fidelity because it avoids unnecessary modifications to unmasked regions. It also states that BVI can suppress mask-shaped semantic bias in models like VACE, improving the success rate of object removal from 62.2% to 97.8% (Wu et al., 30 Apr 2026).
6. Relation to earlier variable-length indexing and compression concepts
BVI should not be conflated with earlier uses of variable-length representation in indexing, compression, or retrieval. A directly relevant contrast appears in “On Optimally Partitioning Variable-Byte Codes” (Pibiri et al., 2018). That paper is not a “Batch Variable-length Indexing” system in the sense of introducing a new batch-oriented indexing architecture. Instead, it proposes an optimal partitioning strategy for inverted lists that can be applied to a point-wise encoder, with Variable-Byte as the main example. Its mechanism partitions each posting list into variable-sized blocks and encodes each block with whichever of two representations is cheaper: the characteristic bit-vector for dense blocks or the point-wise encoder for sparse blocks. It is therefore a posting-list compression method based on within-list partitioning and block-wise adaptive encoding selection, not a differentiable batchwise token-indexing operator.
A second comparison point is “Variable-Length Hashing” (Yu et al., 2016). That work separates how a hash code is represented for storage from how it is used for search. It compresses redundant fixed-length hash codes losslessly into variable-length codes while preserving efficient nearest-neighbor lookup through a multiple hash table structure. The method partitions a fixed-length hash code into substrings, encodes each substring separately, stores variable-length codes in the buckets, and uses the original fixed-length substrings as lookup keys. In the same framework, Block K-means Hashing deliberately adds redundancy to improve retrieval and then removes that redundancy for storage. This is a variable-length representation for large-scale similarity search, not a mask-conditioned operator for variable-length token processing in a DiT.
These comparisons sharpen the meaning of BVI. In YOSE, BVI is a dynamic, differentiable indexing mechanism whose primary object is a masked latent token set and whose primary constraint is variable-length batching across samples (Wu et al., 30 Apr 2026). In the earlier works, variable-length structure appears as adaptive compression or storage-efficient retrieval. This suggests that the common phrase “variable-length indexing” spans distinct research traditions: posting-list partitioning, variable-length hashing, and batchwise differentiable token selection. BVI belongs specifically to the last of these.
