Convolutional Kernel Modulator (CKM) Mechanisms

• CKM is a mechanism that modulates convolutional kernels by either resizing with fixed interpolation matrices or using learned small MLPs, depending on the application.
• In patch-based PDE surrogates, CKM enables compute-adaptivity by allowing dynamic inference-time patch size changes, trading computational cost for accuracy.
• In ConvNets, CKM facilitates parameter-efficient transfer learning and meta-learning by adapting frozen kernel backbones with minimal trainable modulators.

Searching arXiv for the specified papers and closely related work to ground the article in the cited literature. Convolutional Kernel Modulator (CKM) denotes a kernel-modulation mechanism in which the effective convolutional weights used at execution time are derived from a learned or frozen base kernel rather than applied directly. In the recent arXiv literature, the designation appears in two distinct settings. In patch-based transformer surrogates for time-dependent partial differential equations (PDEs), CKM is a lightweight, architecture-agnostic module that dynamically resizes the convolutional patching kernel so that patch size can be selected at inference time without retraining. In ConvNets for transfer learning and meta-learning, kernel modulation is a parameter-efficient method in which small task-specialized modulators transform frozen convolutional kernels into task-adapted kernels while leaving the backbone weights unchanged (Mukhopadhyay et al., 12 Jul 2025, Hu et al., 2022).

1. Terminological scope and conceptual core

Both uses of CKM share a common operational idea: convolution is performed with a modulated kernel $\tilde W$ or $W'$ rather than the original kernel $W$. The modulated kernel is constructed so that the surrounding architecture can remain largely intact. This commonality is structural rather than objective-driven. In the PDE-surrogate setting, the purpose is compute elasticity: the encoding and decoding patch size can be changed at inference in order to trade off compute versus accuracy. In the ConvNet setting, the purpose is parameter efficiency: a frozen base network is adapted to new tasks by training only small kernel modulators and the task head (Mukhopadhyay et al., 12 Jul 2025, Hu et al., 2022).

A concise comparison clarifies the distinction.

Context	What is modulated	Primary objective
Patch-based PDE surrogates	The learned base convolutional kernel is resized through a differentiable projection using fixed interpolation matrices	Inference-time patch-size tunability and compute-adaptive rollout
ConvNets for transfer learning and meta-learning	Frozen convolutional weights are transformed by a small two-layer MLP with tanh	Parameter-efficient task adaptation

A common source of confusion is to treat CKM as a single standardized module. The literature instead uses the label for two technically different mechanisms that operate on convolutional kernels under different optimization regimes and deployment constraints. One changes spatial compression in patchers and un-patchers; the other changes task specialization in frozen ConvNet backbones.

2. CKM in patch-based transformer surrogates for PDEs

In modern patch-based transformer surrogates for time-dependent PDEs, spatial downsampling is implemented by a small convolutional encoder or patcher, followed by a large transformer processor and then a symmetric convolutional decoder or un-patcher. In most ViT-style surrogates, the patch size is fixed at training time, often $k=s=16$, and this fixed choice determines both representational fidelity and inference cost. CKM was introduced to make this encoding and decoding layer compute-elastic: at inference time one can select patch size $k\in\{4,8,16\}$ without retraining, thereby moving along a compute-accuracy trade-off curve (Mukhopadhyay et al., 12 Jul 2025).

Architecturally, CKM is inserted at each convolutional downsampling stage in both the encoder and decoder. In a two-stage hMLP-style patcher, the first convolutional layer maps the full-resolution field $x\in\mathbb{R}^{B\times H\times W\times C}$ to tokens of spatial resolution $H/s\times W/s$. Under CKM, this convolution uses a resized kernel $W'$ and stride $s=k$. After the transformer processor yields latent tokens $x_{\mathrm{lat}}$, the decoder’s convolutional transpose also uses $W'$0 with the same stride $W'$1 to reconstruct the predicted field $W'$2. The module is described as light-weight and plug-and-play, and its applicability is stated to extend to vanilla, axial, and Swin transformers as well as hybrid architectures such as CViT, because the transformer core itself is not altered (Mukhopadhyay et al., 12 Jul 2025).

The central deployment consequence is that token count changes with patch size according to $W'$3. Smaller patches therefore increase token count and compute while improving fidelity; larger patches reduce compute at some accuracy cost. This suggests a direct mechanism for budget-conscious deployment in production environments where inference cost is a primary constraint.

3. Kernel resizing, training regime, and inference workflow

The PDE formulation begins with a base convolutional kernel $W'$4 learned under a fixed patch size $W'$5. At each forward pass, CKM samples a patch size $W'$6 and constructs an interpolation matrix $W'$7, for example via bicubic interpolation, together with its pseudoinverse $W'$8. The modulation function is defined by projection through this resize operator,

$W'$9

and the final modulated kernel is written as

$W$0

The intent is to ensure that features extracted under smaller or larger patches remain aligned with the weights learned under the base patch size (Mukhopadhyay et al., 12 Jul 2025).

The training and inference workflow is correspondingly simple. During training, patch sizes $W$1 are sampled uniformly, or under some schedule, so that the model learns to operate under all three patch resolutions. The loss, whether for next-step prediction or rollout fidelity, is computed on the reconstructed fields, and gradients flow through the convolutional kernels $W$2. No separate training of modulation matrices is required, because the only new objects, $W$3 and $W$4, are fixed interpolation matrices rather than learned parameters. At inference, one chooses a desired patch size $W$5, or alternates it cyclically from step to step, and applies the same modulation in encoder and decoder without fine-tuning or retraining (Mukhopadhyay et al., 12 Jul 2025).

The core pseudocode reported for the method is: $x_{\mathrm{lat}}$4 A frequent misconception is that CKM introduces additional learned modulation parameters analogous to adapters. In this PDE formulation, it does not: the base kernel is shared across all $W$6, and the resize operators are fixed.

4. Empirical behavior in PDE rollouts and relation to CSM

CKM naturally complements the Convolutional Stride Modulator (CSM), which holds the kernel $W$7 fixed and varies only the stride $W$8. Together, CKM and CSM are presented as a fully compute-elastic framework that trades token count for accuracy without touching the transformer core or the training regime. Empirically, CKM-enabled models match or exceed the accuracy of three separately trained fixed-patch models at patch sizes $W$9 across challenging $k=s=16$0D and $k=s=16$1D PDE datasets including shear, turbulent, active, Rayleigh–Bénard, and supernova (Mukhopadhyay et al., 12 Jul 2025).

Figure 1 is reported to show that, as token count increases and patch size decreases, the variance-scaled root-MSE (VRMSE) falls monotonically. CKM follows this accuracy curve of fixed-patch baselines without extra training. In long-horizon rollouts, CKM models exhibit significantly fewer patch artifacts, described as grid-aligned oscillations that appear as harmonic spectral spikes, particularly when patch size is cycled from one time step to the next. Table 5 reports up to a $k=s=16$2 improvement in 10-step rollout VRMSE on the shear dataset relative to a fixed-patch model (Mukhopadhyay et al., 12 Jul 2025).

The ablation results delimit the valid operating regime. Training must include all desired $k=s=16$3 values in order to generalize across them. The choice of base patch size $k=s=16$4 has only mild effect. Alternating patch schedules outperform random schedules during rollout. A plausible implication is that the benefit of cyclic patch-size rollout is not only compute adaptivity but also artifact mitigation through structured variation in the patching lattice.

5. CKM as parameter-efficient kernel modulation in ConvNets

A distinct use of the CKM label appears in parameter-efficient training of ConvNets. The starting point is an ordinary ConvNet such as ResNet-32 or ResNet-50. All convolution layers retain their usual weight tensors $k=s=16$5, but these kernels are frozen once the backbone is chosen. Immediately before each convolution, a small kernel modulator $k=s=16$6 takes the frozen $k=s=16$7 and produces a modulated version $k=s=16$8, after which the network computes

$k=s=16$9

Normalization layers, either BatchNorm or GroupNorm, are retained and treated as an implicit form of modulation whose affine parameters may also be trained (Hu et al., 2022).

The mathematical formulation reshapes each convolutional tensor into a two-dimensional matrix. For $k\in\{4,8,16\}$0, one first reshapes to $k\in\{4,8,16\}$1, applies a layer-specific modulator, and reshapes back: $k\in\{4,8,16\}$2 The published design uses exactly a two-layer MLP with tanh. Let $k\in\{4,8,16\}$3 and let $k\in\{4,8,16\}$4 comprise $k\in\{4,8,16\}$5 with no biases. Then

$k\in\{4,8,16\}$6

so each of the $k\in\{4,8,16\}$7 rows of $k\in\{4,8,16\}$8 is passed independently through the same $k\in\{4,8,16\}$9 MLP. Initialization is chosen so that $x\in\mathbb{R}^{B\times H\times W\times C}$0 at the start: $x\in\mathbb{R}^{B\times H\times W\times C}$1 This makes the modulator approximately the identity map initially (Hu et al., 2022).

The implicit modulation viewpoint arises from normalization. BatchNorm or GroupNorm can be algebraically rewritten as

$x\in\mathbb{R}^{B\times H\times W\times C}$2

which the paper interprets as a multiplicative-and-additive modulation of $x\in\mathbb{R}^{B\times H\times W\times C}$3. In this framework, explicit modulation is the MLP-based transform $x\in\mathbb{R}^{B\times H\times W\times C}$4, while implicit modulation is the training of the affine normalization parameters $x\in\mathbb{R}^{B\times H\times W\times C}$5 (Hu et al., 2022).

6. Parameter counts, training settings, and reported results in ConvNets

The parameter-efficiency argument is explicit. A convolutional layer with $x\in\mathbb{R}^{B\times H\times W\times C}$6 filters of size $x\in\mathbb{R}^{B\times H\times W\times C}$7 has $x\in\mathbb{R}^{B\times H\times W\times C}$8 frozen parameters, while its modulator has only $x\in\mathbb{R}^{B\times H\times W\times C}$9 trainable parameters. The paper gives a ResNet-32 example: a convolution with $H/s\times W/s$0 kernels of $H/s\times W/s$1 over $H/s\times W/s$2 channels has $H/s\times W/s$3 frozen parameters, whereas the modulator has $H/s\times W/s$4 trainable parameters. Summed over all convolutional layers of ResNet-50, the modulators amount to roughly $H/s\times W/s$5K trainable parameters versus $H/s\times W/s$6M frozen parameters, approximately $H/s\times W/s$7 of the base network (Hu et al., 2022).

The same paper reports three training regimes. On CIFAR-10 with ResNet-32 from scratch, CKM with BN, explicit modulation, and classifier trains $H/s\times W/s$8K parameters and reaches $H/s\times W/s$9 accuracy, compared with $W'$0 for full training using $W'$1K trainable parameters. On transfer learning with ImageNet-pretrained ResNet-50, all convolution kernels are frozen, BatchNorm may be replaced by GroupNorm when batch size is small, and the trainable parameters are the modulators $W'$2, normalization $W'$3, and the final linear classifier. On Aircraft, Cars, Flowers, and Food, trained for $W'$4 epochs with batch size $W'$5 and cross-entropy loss, the reported accuracies are $W'$6 on Aircraft, $W'$7 on Cars, $W'$8 on Flowers, and $W'$9 on Food, using about $s=k$0 of the ResNet-50 weights (Hu et al., 2022).

In meta-learning, the inner loop adapts only the modulators and the task’s linear head while keeping all convolutional $s=k$1 frozen, and the outer loop updates the initial modulator parameters by differentiating through inner-loop adaptation. On $s=k$2-way Omniglot and $s=k$3-way mini-ImageNet tasks at $s=k$4- and $s=k$5-shot, CKM is reported to beat ANIL by about $s=k$6 on Omniglot and to tie or slightly exceed MAML on mini-ImageNet, with fewer than $s=k$7 of the total parameters trainable. For deployment, the paper contrasts naive storage for $s=k$8 customized ResNet-50 tasks, $s=k$9 GB, with CKM storage of approximately $x_{\mathrm{lat}}$0 MB total, corresponding to a $x_{\mathrm{lat}}$1 saving. The additional computation is described as one extra two-layer MLP per channel, negligible in wall-clock terms on modern hardware for small kernels such as $x_{\mathrm{lat}}$2 (Hu et al., 2022).

7. Interpretation, limits, and recurrent misconceptions

Across both lines of work, CKM should not be reduced to a generic synonym for “adapters for convolution.” In the PDE-surrogate formulation, CKM does not add learned adapter weights; instead, it resizes a shared base patching kernel through fixed interpolation operators and thereby permits inference-time patch-size control. In the ConvNet formulation, the modulator is explicitly learned and task-specific, and its purpose is to adapt a frozen backbone with minimal trainable state. The two mechanisms therefore share the act of kernel modulation but differ in whether modulation is primarily geometric and deployment-time or primarily task-adaptive and training-time (Mukhopadhyay et al., 12 Jul 2025, Hu et al., 2022).

Another recurrent misconception is that modulation alone guarantees arbitrary generalization outside the training configuration. The PDE ablations state that training must include all desired patch sizes to generalize across them. Likewise, the ConvNet method does not update the frozen convolutional kernels themselves; its performance depends on the expressive sufficiency of the small $x_{\mathrm{lat}}$3 MLPs and the trained normalization parameters. These constraints indicate that CKM is best understood as a controlled mechanism for altering the effective kernel under a specified regime, not as a universal substitute for retraining.

Taken together, the two usages establish a broader research pattern: kernel modulation can be employed either to re-parameterize spatial compression in patch-based surrogate modeling or to re-parameterize task adaptation in frozen convolutional backbones. This suggests a unifying view in which the convolutional kernel is treated as an intermediate object that can be transformed to satisfy system-level goals such as compute adaptivity, parameter efficiency, rollout stability, or multi-task deployment, while preserving the surrounding architectural scaffold.