Cyclic Patch-Size Rollout in PDE Surrogates

• The paper introduces cyclic patch-size rollout to alternate patch sizes (e.g., {4,8,16}) at inference, effectively mitigating grid-induced checkerboard artifacts in PDE surrogate modeling.
• It employs lightweight, architecture-agnostic modules like CKM and CSM to dynamically adjust patch extraction without modifying the transformer backbone.
• Empirical results show up to 50% VRMSE improvements and enhanced long-term stability across various 2D and 3D PDE benchmarks.

Searching arXiv for the specified paper and closely related context. arXiv search query: (Mukhopadhyay et al., 12 Jul 2025) Cyclic patch-size rollout is an inference-time scheduling mechanism for patch-based surrogate models of partial differential equations in which the patch size, or the effective stride that determines patch extraction, alternates across rollout steps rather than remaining fixed. In “Controllable Patching for Compute-Adaptive Surrogate Modeling of Partial Differential Equations,” the mechanism is introduced as part of a framework for inference-time patch-size tunability in patch-based PDE surrogates, implemented through the Convolutional Kernel Modulator (CKM) and Convolutional Stride Modulator (CSM), and combined with a cyclic schedule such as $\{4,8,16\}$ to mitigate “checkerboard” or harmonic artifacts in long autoregressive rollouts without retraining (Mukhopadhyay et al., 12 Jul 2025).

1. Definition and Operational Setting

In a standard autoregressive rollout of a patch-based ViT surrogate, the patch or stride size is fixed once at the start and then kept constant at every timestep. The cited work identifies this as a major source of long-horizon degradation: fixed grids produce “checkerboard” or harmonic artifacts that grow over long rollouts because errors align with the same spatial locations at every step (Mukhopadhyay et al., 12 Jul 2025).

Cyclic patch-size rollout replaces that fixed-grid regime with an alternating schedule. At inference time, and without any retraining, the patch size in CKM or the stride in CSM cycles through a small set of values, one value per time step. For a cycle $P=\{p_1,p_2,p_3\}=\{4,8,16\}$, the stepwise selection rule is

$P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$

so that

$P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$

This schedule continually shifts the grid boundaries. The stated rationale is that systematic error accumulation at the same spatial locations is thereby prevented (Mukhopadhyay et al., 12 Jul 2025).

The mechanism is presented specifically for patch-based transformer surrogates used in spatiotemporal PDE modeling, including 2D and 3D PDE benchmarks and video-like prediction tasks. The transformer block itself, whether vanilla, axial, or other, is unchanged; cyclicity is introduced through the patching interface rather than through modification of the latent processor (Mukhopadhyay et al., 12 Jul 2025).

2. Mathematical Formulation of the Rollout Schedule

Let $x_0$ denote the initial field, and let the goal be to predict $x_t$ for $t=1,\dots,T$ in an autoregressive manner. The paper defines a length-$M$ cycle of patch sizes

$$\{P_t\}_{t=1}^T,\quad P_t = c_{((t-1) \bmod M)+1},\quad c = [4,8,16],\;M=3.$$

At each rollout step $t$, the surrogate applies either CKM or CSM with patch or stride size $P=\{p_1,p_2,p_3\}=\{4,8,16\}$0 (Mukhopadhyay et al., 12 Jul 2025).

For CKM, the base convolutional weight is

$P=\{p_1,p_2,p_3\}=\{4,8,16\}$1

and the resized kernel is obtained through a PI-resize construction based on a bicubic interpolation matrix

$P=\{p_1,p_2,p_3\}=\{4,8,16\}$2

and its pseudoinverse $P=\{p_1,p_2,p_3\}=\{4,8,16\}$3: $P=\{p_1,p_2,p_3\}=\{4,8,16\}$4 The encoder and decoder then use the step-dependent stride $P=\{p_1,p_2,p_3\}=\{4,8,16\}$5: $P=\{p_1,p_2,p_3\}=\{4,8,16\}$6

$P=\{p_1,p_2,p_3\}=\{4,8,16\}$7

For CSM, the kernel size remains fixed at $P=\{p_1,p_2,p_3\}=\{4,8,16\}$8, but the stride is sampled dynamically as $P=\{p_1,p_2,p_3\}=\{4,8,16\}$9: $P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$0

$P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$1

A compact way to distinguish the two mechanisms is to note that CKM changes the kernel and stride jointly through kernel resizing, whereas CSM leaves the kernel untouched and changes only the stride. In both cases, the same cyclic schedule $P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$2 supplies the stepwise control variable (Mukhopadhyay et al., 12 Jul 2025).

3. CKM and CSM as Architecture-Agnostic Modulators

The paper introduces CKM and CSM as lightweight, architecture-agnostic modules that enable dynamic patch size control at inference in patch-based models, without retraining or accuracy loss. Their plug-and-play role is central: rather than replacing the transformer backbone, they adapt the tokenization or patch-extraction interface around it (Mukhopadhyay et al., 12 Jul 2025).

Module	What changes with $P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$3	Core formula
CKM	Kernel size and stride	$P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$4
CSM	Stride only	$P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$5
Shared property	Transformer block unchanged	Same cyclic schedule $P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$6

CKM uses bicubic interpolation and a pseudoinverse-based projection to resize a base kernel of size $P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$7 to the current patch size $P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$8. CSM instead preserves the base kernel and treats the rollout variable as a stride control. The formal distinction matters because the paper applies both variants across multiple backbones and benchmarks, while emphasizing that the transformer block itself is not altered (Mukhopadhyay et al., 12 Jul 2025).

The framework is also described as broadly applicable across architectures. The text explicitly mentions ViTs and AFNO, and notes that in hybrid models such as CViT with a query-based decoder, only CKM is applied to the encoder; even in that restricted configuration, cyclic CKM improves rollouts. This suggests that the cyclic mechanism is not tied to a single token mixer or decoder topology, but to the spatial discretization imposed at the encoder-decoder boundary (Mukhopadhyay et al., 12 Jul 2025).

4. Empirical Behavior: Artifact Suppression and Stability

The principal empirical claim is that cyclic rollout suppresses spectral artifacts and improves long-term rollout fidelity. Figure 1 is described as plotting residual power spectra at rollout step 20 on the turbulent dataset: fixed-p$P_t \;=\; p_{\bigl((t-1)\bmod 3\bigr)+1},$9 shows clear harmonics at frequencies multiple of $P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$0, whereas CKM and CSM with cyclic rollout largely eliminate these spikes (Mukhopadhyay et al., 12 Jul 2025).

Quantitatively, the paper reports 10-step VRMSE reductions of $P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$1–$P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$2 over the corresponding fixed patch 16 model across six diverse PDE benchmarks and two transformer backbones, Axial and Vanilla. A specific example is given on the shear dataset with Vanilla ViT: fixed $P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$3 gives $P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$4, while CKM cyclic gives $P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$5, described as an approximately $P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$6 gain (Mukhopadhyay et al., 12 Jul 2025).

The work also emphasizes visual stability in long rollouts. Figure 2 is summarized as showing a 100-step trajectory of the shear field in which fixed $P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$7 develops checkerboard artifacts by step 40, whereas CKM and CSM remain smooth. In the terminology of the paper, cyclic patch-size rollout improves long-term stability for video-like prediction tasks and improves rollout fidelity and runtime efficiency on a range of challenging 2D and 3D PDE benchmarks (Mukhopadhyay et al., 12 Jul 2025).

A plausible implication is that the benefit arises less from any single preferred patch size than from the repeated displacement of grid boundaries over time. The paper’s own phrasing supports this interpretation by linking the mechanism to prevention of systematic error accumulation at the same spatial locations.

5. Ablations, Failure Modes, and Misconceptions

The paper includes ablations that distinguish cyclic schedules from superficially similar alternatives. The most direct is the comparison with randomized patch-size selection. Section F.3 reports that randomizing patch sizes instead of cycling degrades performance: on the turbulent dataset, cyclic CSM yields $P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$8 versus randomized $P_1=4,\;P_2=8,\;P_3=16,\;P_4=4,\dots$9 (Mukhopadhyay et al., 12 Jul 2025). This directly contradicts a common simplification that any non-fixed patch schedule should be equally effective.

The stated explanation is that a structured cyclic schedule provides consistent spectral smoothing, whereas fully random schedules break periodicity but can re-introduce misalignments each step. The distinction is important because both approaches vary the grid, but only one is presented as systematically improving long-horizon rollout behavior (Mukhopadhyay et al., 12 Jul 2025).

A second failure mode concerns training diversity. Section F.2 reports that omitting a patch size during training, for example never training on $x_0$0, leads to poor results when that size appears at inference. The paper gives the example that VRMSE jumps from approximately $x_0$1 for CKM on turbulent under such omission (Mukhopadhyay et al., 12 Jul 2025). This constrains interpretation of the phrase “without any retraining”: the cyclic rollout schedule itself is inference-time, but all patch sizes intended for rollout must be seen during training through randomized patch sampling.

A further misconception would be to treat cyclic patch-size rollout as a modification of the transformer proper. The paper explicitly states the opposite: the transformer block itself is unchanged. The intervention is confined to how spatial fields are patchified and reconstructed at each autoregressive step (Mukhopadhyay et al., 12 Jul 2025).

6. Design Trade-offs, Scope, and Generalization

The cycle used in the paper is $x_0$2, chosen because these are powers of two commonly used in PDE data. The authors note that a longer cycle or non-power-of-two choices may further diversify grid alignments, but at the cost of increased hyperparameter tuning (Mukhopadhyay et al., 12 Jul 2025). This frames cyclic patch-size rollout not as a uniquely determined algorithm, but as a family of structured schedules with a concrete instantiation.

Compute cost varies substantially across patch sizes. The paper states that smaller patches such as $x_0$3 yield approximately $x_0$4 more tokens than $x_0$5, raising self-attention cost from $x_0$6 to $x_0$7. It therefore suggests that in practice one may cycle between $x_0$8 and $x_0$9 for a milder compute swing (Mukhopadhyay et al., 12 Jul 2025). This makes the mechanism compute-adaptive in a literal sense: patch scheduling trades off token count, attention cost, and rollout robustness.

The framework is positioned as the first to enable inference-time patch-size tunability in patch-based PDE surrogates. Its stated scope includes applicability across architectures, but the paper also identifies open territory: although demonstrated on ViTs and AFNO, further work is needed to explore cyclic patching in Swin, CSWin, or foundation PDE models (Mukhopadhyay et al., 12 Jul 2025). This suggests that the mechanism should be understood as a general foundation for compute-adaptive modeling in PDE surrogate tasks rather than a closed design.

In summary, cyclic patch-size rollout is characterized in the source paper as a simple, inference-only scheduling of patch or stride between a small set of values that has no impact on the transformer block, injects grid diversity to suppress checkerboard artifacts, and yields large gains in long-horizon stability across PDE surrogate tasks and architectures, provided that the rollout sizes have been represented during training (Mukhopadhyay et al., 12 Jul 2025).