Block Reshuffling: Concepts and Applications

• Block reshuffling is a structured reordering of coarse data units that improves computational efficiency and statistical performance across diverse applications.
• It leverages constrained move sets to optimize communication schedules, reduce contention, and enhance mixing or indistinguishability in various domains.
• Block reshuffling methods span parallel numerical linear algebra, stochastic optimization, secure shuffling, and industrial logistics, offering actionable performance gains and theoretical insights.

Searching arXiv for the provided topic and cited works to ground the article in current arXiv records. Block reshuffling denotes a family of structured reordering operations performed at block granularity rather than at element granularity. In parallel numerical linear algebra it is the structured, contention-aware permutation of $2$D block-cyclic matrix blocks during processor-grid resizing; in stochastic optimization it is the permutation of contiguous data blocks between epochs; in permutation algorithms it appears as the movement or swapping of strips; in coding and storage it appears as the reordering of full-block query patterns, encrypted blocks, or blocking unit loads; and in probabilistic shuffles it appears as top-$m$-to-random dynamics with growing block size (0706.2146, Nguyen et al., 31 Mar 2026, Roy et al., 20 Oct 2025, Wan et al., 2024, Patel et al., 2017, Feng, 28 Jun 2026). Across these uses, the common object is a constrained permutation whose quality is assessed through contention, variance, reversals, blocking, mixing, or indistinguishability.

1. Conceptual scope and recurring abstractions

The block in block reshuffling is domain-specific. In the ReSHAPE redistribution algorithm it is a matrix block $Mat(x,y)$ in a $2$D block-cyclic layout; in Sorting by Strip Swaps it is a strip, i.e. a maximal increasing substring; in without-replacement SGD it is a contiguous block of indices; in ORBGRAND it is a full-block error pattern over a codeword; in oblivious shuffling it is an encrypted data block; and in dense buffer storage it is a unit load in a LIFO lane (0706.2146, Roy et al., 20 Oct 2025, Nguyen et al., 31 Mar 2026, Wan et al., 2024, Patel et al., 2017, Disselnmeyer et al., 27 Mar 2026).

A compact way to organize the literature is to separate the unit being moved, the admissible move set, and the optimization criterion.

Domain	Block unit	Primary criterion
Resizable MPI computations	$2$D block-cyclic matrix block	contention-free or contention-reduced schedule
Without-replacement SGD	contiguous data block	smaller prefix-gradient variance constants
Strip/block sorting	maximal increasing substring	minimum number of moves or swaps
ORBGRAND / ORAM	error-pattern query / encrypted block	ML-like ordering / obliviousness
Dense buffers and line pushes	unit load / token	feasibility, distance, or reachable permutations

This suggests that block reshuffling is less a single technique than a structural pattern: coarse units are moved under a restricted dynamics, and the central problem is to factor a global permutation into admissible steps with favorable complexity or information properties. In some papers the blocks are explicit physical or data objects; in others they are induced units such as resampling splits or bucketed gradients (Malinovsky et al., 2022, Nagler et al., 2024, Akitaya et al., 2022).

2. Multidimensional redistribution in resizable parallel computation

In "Efficient Multidimensional Data Redistribution for Resizable Parallel Computations" block reshuffling is the structured, contention-aware permutation of $2$D block-cyclic matrix blocks when an application is resized from a $P_r\times P_c$ processor grid to a $Q_r\times Q_c$ grid (0706.2146). ReSHAPE allows long-running MPI programs to expand or shrink at runtime, and the redistribution problem is to map each block $Mat(x,y)$ from its old owner to its new owner: $$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$

$m$0

The paper exploits periodicity through the superblock dimensions

$m$1

A superblock is the smallest $m$2 submatrix of blocks for which the source-to-destination mapping repeats. On one superblock, the algorithm builds the $m$3 tables

$m$4

$m$5

encoding the initial and final data-processor configurations.

The core schedule is the $m$6 table. Its columns are source processors, its rows are communication steps, and the number of rows is

$m$7

Traversing $m$8 in row-major order and appending each destination to the next free slot of the corresponding source column yields a global communication schedule in which every source sends one message per step. When $m$9 and $Mat(x,y)$0, the schedule is contention-free; in that case the inverse receive table $Mat(x,y)$1 is well defined and each step is a partial permutation with no repeated destination (0706.2146).

When those inequalities do not hold, the raw schedule may exhibit node contention. The paper introduces a Processor Mapping table, initially a copy of $Mat(x,y)$2, and applies grouped circular shifts to spread collisions across steps. If $Mat(x,y)$3 and $Mat(x,y)$4, rows are circularly right-shifted by $Mat(x,y)$5 within each $Mat(x,y)$6-row group. If $Mat(x,y)$7 and $Mat(x,y)$8, columns are circularly down-shifted by $Mat(x,y)$9 within each $2$0-column group. If both $2$1 and $2$2, both transformations are composed. The final block positions are unchanged; what changes is the order in which blocks are sent, so the reshuffling acts on the schedule rather than on the abstract mapping.

The complexity claims are explicit. Total data movement is $2$3 blocks, which is optimal for an $2$4 block matrix. The number of sends in the contention-free schedule is $2$5, independent of $2$6 for fixed processor grids. Empirically, the algorithm is reported as $2$7 faster than Caterpillar for redistribution from $2$8 to $2$9 and $2$0 faster for $2$1; it also reduces MPI communication calls from $2$2 to $2$3 in the $2$4 case and from $2$5 to $2$6 in the $2$7 case (0706.2146). In this literature, block reshuffling is therefore a schedule-synthesis problem for a highly structured permutation.

3. Learning, stochastic optimization, and resampling

In "Learning to Shuffle: Block Reshuffling and Reversal Schemes for Stochastic Optimization" block reshuffling is a structured shuffling strategy for without-replacement SGD that partitions the dataset into contiguous blocks and permutes the blocks, not individual samples, between epochs (Nguyen et al., 31 Mar 2026). For block size $2$8, the indices $2$9 are partitioned into

$2$0

and a block permutation $2$1 produces $2$2. The special cases are exact: $2$3 reduces to random reshuffling, while $2$4 reduces to incremental gradient.

The theoretical object is the prefix average gradient deviation

$2$5

With equal block size $2$6, the block-averaged gradients

$2$7

induce the decomposition

$2$8

hence

$2$9

with strict inequality whenever the within-block variance is positive (Nguyen et al., 31 Mar 2026). The paper’s claim is therefore not merely heuristic: block reshuffling strictly reduces prefix-gradient variance constants under mild conditions. The accompanying paired reversal construction symmetrizes the epoch map and reduces order sensitivity from quadratic to cubic in the step size. Numerical experiments with APR are reported on convex classification, convex regression, and nonconvex benchmarks.

A different block interpretation appears in federated random reshuffling. "Federated Random Reshuffling with Compression and Variance Reduction" treats each client’s local dataset as a block in a two-level finite sum,

$P_r\times P_c$0

and each communication round performs a full reshuffled pass over each client block (Malinovsky et al., 2022). The paper introduces compressed FedRR and two variance-reduced extensions, one for compression noise and one for reshuffling variance. The analysis states that the compression-variance mechanism eliminates dependence on the compression parameter in the asymptotic neighborhood term, and that controlled linear perturbations eliminate variance at the optimum.

The same design axis also appears in example ordering. "GraB: Finding Provably Better Data Permutations than Random Reshuffling" does not define block reshuffling explicitly, but it formalizes permutation design through herding and discrepancy minimization and proves $P_r\times P_c$1 convergence on smooth, non-convex objectives, compared with the $P_r\times P_c$2 rate quoted for random reshuffling (Lu et al., 2022). This suggests a block-level analogue in which blocks, rather than individual examples, are balanced by prefix discrepancy.

Reshuffling also enters hyperparameter optimization through resampling blocks. "Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimization" distinguishes fixed paired splits,

$P_r\times P_c$3

from reshuffled splits, where $P_r\times P_c$4 is redrawn for each configuration (Nagler et al., 2024). In the asymptotic covariance model, reshuffled holdout changes the cross-configuration factor from $P_r\times P_c$5 to $P_r\times P_c$6, whereas reshuffled $P_r\times P_c$7-fold CV has no asymptotic effect. Empirically, the paper reports that reshuffling often improves the final model’s generalization performance on unseen data, drastically improves results for a single train-validation holdout protocol, and can often make holdout become competitive with standard CV while being computationally cheaper.

4. Combinatorial rearrangement, sorting, and mixing

In permutation sorting, block reshuffling is literal. "Sorting by Strip Swaps is NP-Hard" identifies

$P_r\times P_c$8

and studies block moves and strip swaps as primitive reshuffling operations (Roy et al., 20 Oct 2025). The disorder measure is the number of decreasing adjacencies, $P_r\times P_c$9. For Block Sorting, the lower bound is

$Q_r\times Q_c$0

and for Sorting by Strip Swaps,

$Q_r\times Q_c$1

The paper’s reduction replaces each decreasing adjacency with a cage $Q_r\times Q_c$2, in which the only decreases are the two internal ones, and uses hinge gadgets to couple adjacent cages. The resulting equivalence is

$Q_r\times Q_c$3

which yields NP-hardness of Sorting by Strip Swaps (Roy et al., 20 Oct 2025). In this setting, block reshuffling is a constrained permutation problem whose hardness survives even when the moved units are already-sorted strips.

Probabilistic block reshuffling appears in "Cutoff profiles for colored top-$Q_r\times Q_c$4-to-random shuffles with growing block size" (Feng, 28 Jun 2026). One step removes the top $Q_r\times Q_c$5 cards, recolors them independently, permutes them within the block, and inserts the block into uniformly random positions. The obstruction to mixing is the untouched-label count

$Q_r\times Q_c$6

with

$Q_r\times Q_c$7

If $Q_r\times Q_c$8 and $Q_r\times Q_c$9, then

$Mat(x,y)$0

Using the exact nested-set reduction, the separation distance converges to

$Mat(x,y)$1

and to

$Mat(x,y)$2

(Feng, 28 Jun 2026). Here the block is a physical segment of the deck, and reshuffling controls cutoff profiles through the coupon-collector clock $Mat(x,y)$3.

A geometric variant is studied in "Pushing Blocks by Sweeping Lines" (Akitaya et al., 2022). The move primitive is a line push from one cardinal direction, displacing all extreme tokens by at most one cell subject to blocking. For sparse configurations of $Mat(x,y)$4 tokens, the paper proves that every sparse configuration can be pushed into an $Mat(x,y)$5 box if and only if

$Mat(x,y)$6

In the labeled compact case, all reachable permutations are even, and except for a small number of degenerate cases the permutation group on non-core tokens is the alternating group (Akitaya et al., 2022). The relevant reshuffling constraints are row and column occupancy, core invariance, and parity.

5. Decoding, obliviousness, and industrial buffer reshuffling

In coding theory, "Approaching Maximum Likelihood Decoding Performance via Reshuffling ORBGRAND" uses block reshuffling to mean a global reorder of the full-block error patterns that ORBGRAND queries over a codeword (Wan et al., 2024). The idealized search problem defines

$Mat(x,y)$7

where $Mat(x,y)$8 is the posterior probability that the $Mat(x,y)$9-th queried pattern corresponds to the true codeword. An optimal schedule must satisfy

$$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$0

RS-ORBGRAND starts from a base ORB-type ordering, computes $$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$1 offline, sorts indices in descending order of $$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$2, and composes this reshuffling with the base schedule. The reported outcome is that at BLER around $$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$3, RS-ORBGRAND outperforms other ORB-type decoders by at least $$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$4 and is within $$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$5 of ML decoding (Wan et al., 2024). In this literature, block reshuffling is not movement of data but reordering of a query list over the whole block.

In secure computation, "CacheShuffle: An Oblivious Shuffle Algorithm Using Caches" studies oblivious shuffling and $$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$6-oblivious shuffling of encrypted server-side blocks (Patel et al., 2017). The adversary may know the initial positions of $$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$7 touched blocks, and the goal is to hide the new allocation after reshuffling. The paper reports several explicit tradeoffs: CacheShuffleRoot uses $$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$8 client memory and $$Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),$$9 blocks of bandwidth; CacheShuffle uses $m$00 client memory with $m$01 bandwidth; KCacheShuffleBasic uses $m$02 client storage and exactly $m$03 blocks of bandwidth; KCacheShuffle uses $m$04 client storage and requires $m$05 blocks of bandwidth; and KCacheShuffleDummy uses $m$06 client storage and $m$07 blocks of bandwidth for $m$08 blocks with dummies (Patel et al., 2017). Here the decisive criterion is indistinguishability of the movement transcript.

Industrial logistics supplies a physical blocking model. "The Multi-AMR Buffer Storage, Retrieval, and Reshuffling Problem" generalizes classical block or container reshuffling to a dense floor-storage buffer operated by multiple AMRs under time windows (Disselnmeyer et al., 27 Mar 2026). Static lanes behave as LIFO stacks, only outermost slots are accessible, and blocking unit loads must be reshuffled to other lanes before retrieval. The paper formulates a Binary Integer Programming model and a hierarchical heuristic that decomposes the problem into A* search for task-level sequence planning and Constraint Programming for multi-robot coordination and scheduling. Because the problem is NP-hard, exact methods are computationally intractable at industrial scale; the experiments report orders-of-magnitude computation time reductions for the heuristic relative to the exact formulation (Disselnmeyer et al., 27 Mar 2026). In this domain, block reshuffling is inseparable from storage assignment, retrieval windows, lane locking, and fleet scheduling.

6. Cross-cutting structure and research directions

Across these literatures, block reshuffling is usually defined by three ingredients that recur explicitly in the papers: a block abstraction, a constrained move set, and a disorder or exposure measure. The block abstraction may be a superblock-relative matrix position, a strip, a contiguous training-data segment, a codeword-level error pattern, a lane-resident unit load, an encrypted server block, or a top-$m$09 card segment. The move set may be a contention-aware message round, a block move, a strip swap, a block permutation, a line push, a lane relocation, or a re-encryption-and-upload step. The disorder measure may be node contention, prefix-gradient variance, $m$10, untouched labels, expected query inversions, blocking depth, or adversarial knowledge of touched blocks (0706.2146, Nguyen et al., 31 Mar 2026, Roy et al., 20 Oct 2025, Feng, 28 Jun 2026, Wan et al., 2024, Patel et al., 2017).

A second recurring feature is the tension between global structure and local operations. ReSHAPE outperforms Caterpillar because it constructs a global communication schedule rather than local rotating exchanges (0706.2146). RS-ORBGRAND improves ORB-type decoding by globally reordering the query list according to average posterior mass (Wan et al., 2024). In stochastic optimization, block reshuffling and paired reversal improve constants and order sensitivity within a unified framework rather than by ad hoc permutations (Nguyen et al., 31 Mar 2026). GraB similarly indicates that better-than-random permutations can be found by discrepancy control, which suggests a broader design space of structured reshuffling rules (Lu et al., 2022).

The open problems are likewise domain-specific but structurally related. In stochastic optimization, optimal block sizing, adaptive regrouping, and fuller end-to-end theory for reversal-based schemes remain open (Nguyen et al., 31 Mar 2026). In federated learning, partial participation, asynchronous execution, and nonconvex objectives remain open for block-wise random reshuffling with compression (Malinovsky et al., 2022). In line-push reconfiguration, the decision complexity of sparse-to-compact reachability and the shortest-solution problem remain open (Akitaya et al., 2022). In dense floor buffers, richer kinematics, stochastic arrivals, and tighter integration of sequence planning with routing remain open (Disselnmeyer et al., 27 Mar 2026). In HPO, extending the analysis from finite candidate sets and asymptotic Gaussian-process models to fully adaptive search remains open (Nagler et al., 2024).

This suggests that block reshuffling is best understood as a unifying research theme about how coarse, already-structured units should be permuted under nontrivial operational constraints. The literature shows that once the block is chosen, the central questions become schedule synthesis, obstruction control, and the design of permutations that are simultaneously feasible, efficient, and, where required, statistically or cryptographically opaque.