---
title: 'Block Reshuffling: Concepts and Applications'
url: https://www.emergentmind.com/topics/block-reshuffling
type: topic
---

# Block Reshuffling: Concepts and Applications

Searching arXiv for the provided topic and cited works to ground the article in current arXiv records.
Block reshuffling denotes a family of structured reordering operations performed at block granularity rather than at element granularity. In parallel numerical linear algebra it is the structured, contention-aware permutation of \(2\)D block-cyclic matrix blocks during processor-grid resizing; in stochastic optimization it is the permutation of contiguous data blocks between epochs; in permutation algorithms it appears as the movement or swapping of strips; in coding and storage it appears as the reordering of full-block query patterns, encrypted blocks, or blocking unit loads; and in probabilistic shuffles it appears as top-\(m\)-to-random dynamics with growing block size [0706.2146] [2604.00260] [2511.00015] [2401.15946] [1705.07069] [2606.29530]. Across these uses, the common object is a constrained permutation whose quality is assessed through contention, variance, reversals, blocking, mixing, or indistinguishability.

## 1. Conceptual scope and recurring abstractions

The block in block reshuffling is domain-specific. In the ReSHAPE redistribution algorithm it is a matrix block \(Mat(x,y)\) in a \(2\)D block-cyclic layout; in Sorting by Strip Swaps it is a strip, i.e. a maximal increasing substring; in without-replacement SGD it is a contiguous block of indices; in ORBGRAND it is a full-block error pattern over a codeword; in oblivious shuffling it is an encrypted data block; and in dense buffer storage it is a unit load in a LIFO lane [0706.2146] [2511.00015] [2604.00260] [2401.15946] [1705.07069] [2603.26542].

A compact way to organize the literature is to separate the unit being moved, the admissible move set, and the optimization criterion.

| Domain | Block unit | Primary criterion |
|---|---|---|
| Resizable MPI computations | \(2\)D block-cyclic matrix block | contention-free or contention-reduced schedule |
| Without-replacement SGD | contiguous data block | smaller prefix-gradient variance constants |
| Strip/block sorting | maximal increasing substring | minimum number of moves or swaps |
| ORBGRAND / ORAM | error-pattern query / encrypted block | ML-like ordering / obliviousness |
| Dense buffers and line pushes | unit load / token | feasibility, distance, or reachable permutations |

This suggests that block reshuffling is less a single technique than a structural pattern: coarse units are moved under a restricted dynamics, and the central problem is to factor a global permutation into admissible steps with favorable complexity or information properties. In some papers the blocks are explicit physical or data objects; in others they are induced units such as resampling splits or bucketed gradients [2205.03914] [2405.15393] [2202.12045].

## 2. Multidimensional redistribution in resizable parallel computation

In "Efficient Multidimensional Data Redistribution for Resizable Parallel Computations" block reshuffling is the structured, contention-aware permutation of \(2\)D block-cyclic matrix blocks when an application is resized from a \(P_r\times P_c\) processor grid to a \(Q_r\times Q_c\) grid [0706.2146]. ReSHAPE allows long-running MPI programs to expand or shrink at runtime, and the redistribution problem is to map each block \(Mat(x,y)\) from its old owner to its new owner:
\[
Source(x,y) = P_c \cdot (x \bmod P_r) + (y \bmod P_c),
\]
\[
Dest(x,y) = Q_c \cdot (x \bmod Q_r) + (y \bmod Q_c).
\]

The paper exploits periodicity through the superblock dimensions
\[
R = \operatorname{lcm}(P_r, Q_r), \qquad C = \operatorname{lcm}(P_c, Q_c).
\]
A superblock is the smallest \(R\times C\) submatrix of blocks for which the source-to-destination mapping repeats. On one superblock, the algorithm builds the \(R\times C\) tables
\[
IDPC(i,j)=P_c\cdot(i\bmod P_r)+(j\bmod P_c),
\]
\[
FDPC(i,j)=Q_c\cdot(i\bmod Q_r)+(j\bmod Q_c),
\]
encoding the initial and final data-processor configurations.

The core schedule is the \(C_{Transfer}\) table. Its columns are source processors, its rows are communication steps, and the number of rows is
\[
C_{TransferRows} = \frac{R\cdot C}{P_rP_c}.
\]
Traversing \(FDPC\) in row-major order and appending each destination to the next free slot of the corresponding source column yields a global communication schedule in which every source sends one message per step. When \(P_r \le Q_r\) and \(P_c \le Q_c\), the schedule is contention-free; in that case the inverse receive table \(C_{Recv}\) is well defined and each step is a partial permutation with no repeated destination [0706.2146].

When those inequalities do not hold, the raw schedule may exhibit node contention. The paper introduces a Processor Mapping table, initially a copy of \(FDPC\), and applies grouped circular shifts to spread collisions across steps. If \(P_r > Q_r\) and \(P_c < Q_c\), rows are circularly right-shifted by \(P_c\cdot i\) within each \(P_r\)-row group. If \(P_r < Q_r\) and \(P_c > Q_c\), columns are circularly down-shifted by \(P_r\cdot j\) within each \(P_c\)-column group. If both \(P_r > Q_r\) and \(P_c > Q_c\), both transformations are composed. The final block positions are unchanged; what changes is the order in which blocks are sent, so the reshuffling acts on the schedule rather than on the abstract mapping.

The complexity claims are explicit. Total data movement is \(O(N^2)\) blocks, which is optimal for an \(N\times N\) block matrix. The number of sends in the contention-free schedule is \(R C\), independent of \(N\) for fixed processor grids. Empirically, the algorithm is reported as \(\sim 12.7\times\) faster than Caterpillar for redistribution from \(2\times 4\) to \(5\times 8\) and \(\sim 32\times\) faster for \(8\to 50\); it also reduces MPI communication calls from \(160\) to \(80\) in the \(8\to 40\) case and from \(392\) to \(196\) in the \(8\to 50\) case [0706.2146]. In this literature, block reshuffling is therefore a schedule-synthesis problem for a highly structured permutation.

## 3. Learning, stochastic optimization, and resampling

In "Learning to Shuffle: Block Reshuffling and Reversal Schemes for Stochastic Optimization" block reshuffling is a structured shuffling strategy for without-replacement SGD that partitions the dataset into contiguous blocks and permutes the blocks, not individual samples, between epochs [2604.00260]. For block size \(b\), the indices \(I=(1,2,\dots,n)\) are partitioned into
\[
\mathcal{B}(I;b)=(B_1,\dots,B_K), \qquad B_k=((k-1)b+1,\dots,\min(kb,n)),
\]
and a block permutation \(\sigma\) produces \(\mathrm{BlockShuffle}(I;b,\sigma)=(B_{\sigma(1)},\dots,B_{\sigma(K)})\). The special cases are exact: \(b=1\) reduces to random reshuffling, while \(b=n\) reduces to incremental gradient.

The theoretical object is the prefix average gradient deviation
\[
\Delta_{\pi}^{(k)}(w)=\frac{1}{k}\sum_{t=1}^k g_{\pi(t)}(w)-\nabla F(w).
\]
With equal block size \(b=n/K\), the block-averaged gradients
\[
G_r(w):=\frac{1}{b}\sum_{i\in B_r} g_i(w)
\]
induce the decomposition
\[
\sigma_{\mathrm{ind}}^2(w)=\sigma_{\mathrm{within}}^2(w)+\sigma_{\mathrm{blk}}^2(w),
\]
hence
\[
\sigma_{\mathrm{blk}}^2(w)\le \sigma_{\mathrm{ind}}^2(w),
\]
with strict inequality whenever the within-block variance is positive [2604.00260]. The paper’s claim is therefore not merely heuristic: block reshuffling strictly reduces prefix-gradient variance constants under mild conditions. The accompanying paired reversal construction symmetrizes the epoch map and reduces order sensitivity from quadratic to cubic in the step size. Numerical experiments with APR are reported on convex classification, convex regression, and nonconvex benchmarks.

A different block interpretation appears in federated random reshuffling. "Federated Random Reshuffling with Compression and Variance Reduction" treats each client’s local dataset as a block in a two-level finite sum,
\[
f(x)=\frac{1}{M}\sum_{m=1}^M g_m(x), \qquad g_m(x)=\frac{1}{n}\sum_{i=1}^n f_{m,i}(x),
\]
and each communication round performs a full reshuffled pass over each client block [2205.03914]. The paper introduces compressed FedRR and two variance-reduced extensions, one for compression noise and one for reshuffling variance. The analysis states that the compression-variance mechanism eliminates dependence on the compression parameter in the asymptotic neighborhood term, and that controlled linear perturbations eliminate variance at the optimum.

The same design axis also appears in example ordering. "GraB: Finding Provably Better Data Permutations than Random Reshuffling" does not define block reshuffling explicitly, but it formalizes permutation design through herding and discrepancy minimization and proves \(O(T^{-2/3})\) convergence on smooth, non-convex objectives, compared with the \(O(n^{1/3}T^{-2/3})\) rate quoted for random reshuffling [2205.10733]. This suggests a block-level analogue in which blocks, rather than individual examples, are balanced by prefix discrepancy.

Reshuffling also enters hyperparameter optimization through resampling blocks. "Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimization" distinguishes fixed paired splits,
\[
I_{m,j}=I_m \quad \text{for all } j,
\]
from reshuffled splits, where \(I_{m,j}\) is redrawn for each configuration [2405.15393]. In the asymptotic covariance model, reshuffled holdout changes the cross-configuration factor from \(\tau^2=1\) to \(\tau^2=\alpha\), whereas reshuffled \(M\)-fold CV has no asymptotic effect. Empirically, the paper reports that reshuffling often improves the final model’s generalization performance on unseen data, drastically improves results for a single train-validation holdout protocol, and can often make holdout become competitive with standard CV while being computationally cheaper.

## 4. Combinatorial rearrangement, sorting, and mixing

In permutation sorting, block reshuffling is literal. "Sorting by Strip Swaps is NP-Hard" identifies
\[
\text{block}=\text{strip}=\text{maximal increasing substring that is a substring of the identity permutation}
\]
and studies block moves and strip swaps as primitive reshuffling operations [2511.00015]. The disorder measure is the number of decreasing adjacencies, \(\mathrm{rev}(\pi)\). For Block Sorting, the lower bound is
\[
bs(\pi)\ge \mathrm{rev}(\pi),
\]
and for Sorting by Strip Swaps,
\[
SSD(\pi)\ge \left\lceil \frac{\mathrm{rev}(\pi)}{2}\right\rceil.
\]
The paper’s reduction replaces each decreasing adjacency with a cage \(L_i\,a_i\,m_i\,a_{i+1}\,U_i\), in which the only decreases are the two internal ones, and uses hinge gadgets to couple adjacent cages. The resulting equivalence is
\[
bs(\pi)=R \quad\Longleftrightarrow\quad SSD(\pi^\dagger)=R=\frac{\mathrm{rev}(\pi^\dagger)}{2},
\]
which yields NP-hardness of Sorting by Strip Swaps [2511.00015]. In this setting, block reshuffling is a constrained permutation problem whose hardness survives even when the moved units are already-sorted strips.

Probabilistic block reshuffling appears in "Cutoff profiles for colored top-\(m\)-to-random shuffles with growing block size" [2606.29530]. One step removes the top \(m\) cards, recolors them independently, permutes them within the block, and inserts the block into uniformly random positions. The obstruction to mixing is the untouched-label count
\[
E_{k_n}^{(m_n)}=n-|T_{k_n}|,
\]
with
\[
b_n=n-m_n,\qquad q_n=\frac{b_n}{n},\qquad \lambda_n=nq_n^{k_n}.
\]
If \(\lambda_n\to\lambda\in(0,\infty)\) and \(b_n\to\infty\), then
\[
E_{k_n}^{(m_n)} \Rightarrow \mathrm{Poisson}(\lambda).
\]
Using the exact nested-set reduction, the separation distance converges to
\[
1-e^{-\lambda}(1+\lambda)\quad \text{for } p=1,
\]
and to
\[
1-e^{-\lambda}\quad \text{for } p\ge 2
\]
[2606.29530]. Here the block is a physical segment of the deck, and reshuffling controls cutoff profiles through the coupon-collector clock \(n(1-m_n/n)^{k_n}\).

A geometric variant is studied in "Pushing Blocks by Sweeping Lines" [2202.12045]. The move primitive is a line push from one cardinal direction, displacing all extreme tokens by at most one cell subject to blocking. For sparse configurations of \(n=ab\) tokens, the paper proves that every sparse configuration can be pushed into an \(a\times b\) box if and only if
\[
a\le 2,\qquad b\le 2,\qquad \text{or}\qquad a=b=3.
\]
In the labeled compact case, all reachable permutations are even, and except for a small number of degenerate cases the permutation group on non-core tokens is the alternating group [2202.12045]. The relevant reshuffling constraints are row and column occupancy, core invariance, and parity.

## 5. Decoding, obliviousness, and industrial buffer reshuffling

In coding theory, "Approaching Maximum Likelihood Decoding Performance via Reshuffling ORBGRAND" uses block reshuffling to mean a global reorder of the full-block error patterns that ORBGRAND queries over a codeword [2401.15946]. The idealized search problem defines
\[
Q = \sum_{t=1}^{2^n} t\,\mathbf{E}[S_t],
\]
where \(S_t\) is the posterior probability that the \(t\)-th queried pattern corresponds to the true codeword. An optimal schedule must satisfy
\[
\mathbf{E}[S_1] \ge \mathbf{E}[S_2] \ge \cdots.
\]
RS-ORBGRAND starts from a base ORB-type ordering, computes \(\mathbf{E}[S_t]\) offline, sorts indices in descending order of \(\mathbf{E}[S_t]\), and composes this reshuffling with the base schedule. The reported outcome is that at BLER around \(10^{-6}\), RS-ORBGRAND outperforms other ORB-type decoders by at least \(0.3\,\mathrm{dB}\) and is within \(0.1\,\mathrm{dB}\) of ML decoding [2401.15946]. In this literature, block reshuffling is not movement of data but reordering of a query list over the whole block.

In secure computation, "CacheShuffle: An Oblivious Shuffle Algorithm Using Caches" studies oblivious shuffling and \(K\)-oblivious shuffling of encrypted server-side blocks [1705.07069]. The adversary may know the initial positions of \(K\) touched blocks, and the goal is to hide the new allocation after reshuffling. The paper reports several explicit tradeoffs: CacheShuffleRoot uses \(O(\sqrt{N})\) client memory and \((4+\epsilon)N\) blocks of bandwidth; CacheShuffle uses \(O(S)\) client memory with \(O(N\log_S N)\) bandwidth; KCacheShuffleBasic uses \(O(K)\) client storage and exactly \(2N\) blocks of bandwidth; KCacheShuffle uses \(O(S)\) client storage and requires \(2N+(1+\epsilon)O(K\log_S K)\) blocks of bandwidth; and KCacheShuffleDummy uses \(O(K)\) client storage and \(D+(2+\epsilon)N\) blocks of bandwidth for \(N+D\) blocks with dummies [1705.07069]. Here the decisive criterion is indistinguishability of the movement transcript.

Industrial logistics supplies a physical blocking model. "The Multi-AMR Buffer Storage, Retrieval, and Reshuffling Problem" generalizes classical block or container reshuffling to a dense floor-storage buffer operated by multiple AMRs under time windows [2603.26542]. Static lanes behave as LIFO stacks, only outermost slots are accessible, and blocking unit loads must be reshuffled to other lanes before retrieval. The paper formulates a Binary Integer Programming model and a hierarchical heuristic that decomposes the problem into A* search for task-level sequence planning and Constraint Programming for multi-robot coordination and scheduling. Because the problem is NP-hard, exact methods are computationally intractable at industrial scale; the experiments report orders-of-magnitude computation time reductions for the heuristic relative to the exact formulation [2603.26542]. In this domain, block reshuffling is inseparable from storage assignment, retrieval windows, lane locking, and fleet scheduling.

## 6. Cross-cutting structure and research directions

Across these literatures, block reshuffling is usually defined by three ingredients that recur explicitly in the papers: a block abstraction, a constrained move set, and a disorder or exposure measure. The block abstraction may be a superblock-relative matrix position, a strip, a contiguous training-data segment, a codeword-level error pattern, a lane-resident unit load, an encrypted server block, or a top-\(m\) card segment. The move set may be a contention-aware message round, a block move, a strip swap, a block permutation, a line push, a lane relocation, or a re-encryption-and-upload step. The disorder measure may be node contention, prefix-gradient variance, \(\mathrm{rev}(\pi)\), untouched labels, expected query inversions, blocking depth, or adversarial knowledge of touched blocks [0706.2146] [2604.00260] [2511.00015] [2606.29530] [2401.15946] [1705.07069].

A second recurring feature is the tension between global structure and local operations. ReSHAPE outperforms Caterpillar because it constructs a global communication schedule rather than local rotating exchanges [0706.2146]. RS-ORBGRAND improves ORB-type decoding by globally reordering the query list according to average posterior mass [2401.15946]. In stochastic optimization, block reshuffling and paired reversal improve constants and order sensitivity within a unified framework rather than by ad hoc permutations [2604.00260]. GraB similarly indicates that better-than-random permutations can be found by discrepancy control, which suggests a broader design space of structured reshuffling rules [2205.10733].

The open problems are likewise domain-specific but structurally related. In stochastic optimization, optimal block sizing, adaptive regrouping, and fuller end-to-end theory for reversal-based schemes remain open [2604.00260]. In federated learning, partial participation, asynchronous execution, and nonconvex objectives remain open for block-wise random reshuffling with compression [2205.03914]. In line-push reconfiguration, the decision complexity of sparse-to-compact reachability and the shortest-solution problem remain open [2202.12045]. In dense floor buffers, richer kinematics, stochastic arrivals, and tighter integration of sequence planning with routing remain open [2603.26542]. In HPO, extending the analysis from finite candidate sets and asymptotic Gaussian-process models to fully adaptive search remains open [2405.15393].

This suggests that block reshuffling is best understood as a unifying research theme about how coarse, already-structured units should be permuted under nontrivial operational constraints. The literature shows that once the block is chosen, the central questions become schedule synthesis, obstruction control, and the design of permutations that are simultaneously feasible, efficient, and, where required, statistically or cryptographically opaque.

Source: https://www.emergentmind.com/topics/block-reshuffling