Peripheral Frequency Gating in Spatiotemporal Prediction

• Peripheral Frequency Gating (PFG) is a mechanism that dynamically modulates receptive fields using local spectral cues for adaptive spatiotemporal prediction.
• It fuses multi-scale peripheral responses with learnable center suppression, effectively operating as a spatially adaptive band-pass filter.
• PFG enhances fully convolutional architectures by reducing computational cost through separable convolutions while optimizing context selection in video forecasting.

Peripheral Frequency Gating (PFG) is a frequency-guided peripheral gating mechanism introduced as the core block of PFGNet for spatiotemporal predictive learning. In that formulation, PFG dynamically modulates receptive fields pixel by pixel using local spectral cues, adaptively fuses multi-scale large-kernel peripheral responses with learnable center suppression, and thereby behaves as a spatially adaptive band-pass filter. Its stated purpose is to preserve the efficiency and full parallelism of pure convolutional models while compensating for the rigidity of fixed receptive fields in video-like forecasting settings (Cai et al., 24 Feb 2026).

1. Conceptual basis and motivation

PFG was introduced in response to a specific limitation of standard convolutional spatiotemporal predictive learning models: fixed receptive fields are not well matched to spatially varying motion patterns. The motivating claim is that optimal context size varies across the field of view: smooth background regions need suppression of redundant low-frequency content, motion boundaries and textures need larger contextual integration, and noisy areas should avoid over-amplifying high-frequency artifacts (Cai et al., 24 Feb 2026).

The mechanism is explicitly tied to biological center–surround organization and to frequency-selective signal processing. The paper links PFG to center–surround receptive fields in the retina and visual cortex, emphasizing that such receptive fields respond strongly to contrasts and edges rather than to uniform regions. In this reading, PFG is not merely a dynamic fusion block; it is intended as a learnable spatial frequency filter whose inductive bias resembles antagonistic center–surround processing (Cai et al., 24 Feb 2026).

The core signal-processing intuition is that large kernels capture broader contextual, lower-frequency structure, whereas small central kernels capture more local, higher-pass content. Their difference is described as a ring-shaped or annular band-pass filter. The paper writes this relation in DoG-like form,

$$(H_L - \beta H_S) * x = \text{peripheral response} - \beta \cdot \text{central response}.$$

This suggests that PFG is best understood as a mechanism for selecting and shaping mid-frequency responses rather than as a generic multi-branch convolutional module (Cai et al., 24 Feb 2026).

2. Block structure and mathematical formulation

The PFG block begins from an input feature map

$$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$

It first computes a compact frequency descriptor from three fixed local operators: $$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$ Here $G_x$ and $G_y = G_x^\top$ are Sobel filters for gradient magnitude, $L$ is a Laplacian filter, and $\mathbb{E}_{3\times3}[\cdot]$ is local averaging used to compute variance. These three maps are channel-averaged and concatenated into a three-channel descriptor

$\mathbf{F} \in \mathbb{R}^{3 \times H' \times W'}.$

The descriptor supplies the local spectral cues that drive subsequent gating (Cai et al., 24 Feb 2026).

PFG then computes multi-scale peripheral responses using the kernel set

$\mathcal{K} = \{9, 15, 31\}.$

For each scale $k$,

$$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$0

where $$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$1 is a $$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$2 horizontal depthwise convolution and $$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$3 is a $$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$4 vertical depthwise convolution. A small $$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$5 depthwise convolution $$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$6 extracts the center response, and the suppressed peripheral response is defined as

$$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$7

with $$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$8 channel-wise and learnable for each scale. The paper emphasizes that $$\mathbf{X} \in \mathbb{R}^{C' \times H' \times W'}.$$9, allowing both suppression and enhancement, which is presented as important because feature maps can contain positive and negative activations (Cai et al., 24 Feb 2026).

The gating step maps the spectral descriptor through a $$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$0 convolution,

$$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$1

and uses a per-pixel softmax over scales,

$$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$2

The final output is

$$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$3

In the paper’s interpretation, this weighted fusion makes the receptive field spatially adaptive and frequency-sensitive at each pixel (Cai et al., 24 Feb 2026).

3. Band-pass interpretation in the frequency domain

The paper gives PFG an explicit frequency-response interpretation. If $$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$4 denotes the frequency response of the large kernel at scale $$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$5, and $$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$6 denotes the response of the small center kernel, then the combined branch response is

$$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$7

Under the stated conditions, there exists a mid-frequency annulus

$$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$8

such that

$$\begin{aligned} f_1 &= \sqrt{(G_x * \mathbf{X})^2 + (G_y * \mathbf{X})^2}, \ f_2 &= |L * \mathbf{X}|, \ f_3 &= \mathbb{E}_{3\times3}[\mathbf{X}^2] - \mathbb{E}_{3\times3}[\mathbf{X}]^2. \end{aligned}$$9

This is the formal basis for the claim that each PFG branch behaves like a ring-shaped band-pass filter (Cai et al., 24 Feb 2026).

The supplementary analysis further introduces an SNR criterion for center suppression: $G_x$0 and argues that there exists a finite stationary point $G_x$1 satisfying

$G_x$2

This suggests that learnable center suppression is intended to improve signal quality relative to using the large kernel alone, rather than functioning only as an architectural embellishment (Cai et al., 24 Feb 2026).

A likely misconception is to view PFG merely as multi-scale spatial mixing. The paper’s formulation is narrower: the block is designed so that multi-scale peripheral responses are modulated by localized spectral descriptors and filtered through center suppression, yielding an adaptive band-pass operator rather than an unconstrained scale-fusion mechanism (Cai et al., 24 Feb 2026).

4. Computational design and architectural role

PFG is embedded in PFGNet, a fully convolutional framework for spatiotemporal predictive learning. The stated design goal is to obtain structure-aware spatiotemporal modeling without recurrence or attention. In this respect, PFG is positioned as the main source of adaptive context selection inside an otherwise pure CNN architecture (Cai et al., 24 Feb 2026).

A central implementation choice is the decomposition of each $G_x$3 peripheral kernel into two separable one-dimensional convolutions,

$G_x$4

The paper states that this reduces per-channel computational cost from $G_x$5 to $G_x$6. For $G_x$7, it reports roughly a 15× reduction in kernel parameters and MACs per layer. This decomposition is used not only in PFG but also in MSInit, and it is presented as essential for making very large receptive fields practical in spatiotemporal forecasting (Cai et al., 24 Feb 2026).

The comparative positioning is also explicit. Relative to recurrent models such as ConvLSTM or PredRNN, the fully convolutional design is described as more parallelizable and lower in latency and FLOPs. Relative to hybrid architectures such as SwinLSTM or VMRNN, PFGNet remains in the CNN family while injecting frequency-guided adaptivity into the spatial backbone. Relative to attention-based methods, it seeks useful global or contextual effects through large separable kernels and adaptive suppression/fusion. Relative to methods such as AFNO, DCFormer, or wavelet-based models, it avoids explicit transforms by operating directly in the spatial domain with lightweight local frequency descriptors (Cai et al., 24 Feb 2026).

5. Empirical behavior and ablation evidence

The reported evaluations cover Moving MNIST, TaxiBJ, KTH, and Human3.6M, with additional results on Moving Fashion-MNIST. The paper states that PFGNet delivers SOTA or near-SOTA forecasting performance with substantially fewer parameters and FLOPs (Cai et al., 24 Feb 2026).

Dataset	Reported result	Efficiency note
Moving MNIST	MSE 15.2, SSIM 0.967	best recurrent-free result
TaxiBJ	MSE 0.2881	1.9M parameters, 0.6G FLOPs
KTH	best SSIM, competitive PSNR	especially strong at preserving motion structure and contours
Human3.6M	near-optimal performance	7.3M parameters, 58.3G FLOPs

The ablation results are presented as direct evidence for the PFG design. Removing MSInit degrades performance, indicating that PFG works best when fed diversified multi-scale features. Replacing adaptive softmax gating with fixed-weight fusion performs worse, supporting the claim that pixel-wise scale selection is essential. Single-scale branches underperform the full multi-scale setup, and the best results come from fusing $G_x$8. Center suppression is also isolated as important: $G_x$9 leaves redundant low-frequency background, fixed $G_y = G_x^\top$0 lacks spatial adaptivity, and learnable $G_y = G_x^\top$1 performs best. The paper further reports that tanh is better than sigmoid for $G_y = G_x^\top$2, because tanh allows both positive and negative modulation. Using all three cues—gradient, Laplacian, and local variance—works best, and removing any one hurts performance. It also reports that larger kernels and more PFG blocks generally improve results up to a point, with diminishing returns and some dataset-dependent saturation (Cai et al., 24 Feb 2026).

Taken together, these observations support a specific empirical claim: the reported gains are not attributed only to large kernels or only to scale fusion, but to the joint use of localized spectral cues, per-pixel softmax gating, and learnable center suppression within a multi-scale fully convolutional design (Cai et al., 24 Feb 2026).

6. Terminological scope and acronym ambiguity

The acronym PFG has an established and much older meaning in NMR and MRI: pulsed field gradient. In that literature, PFG refers to gradient-based diffusion encoding in PGSE or PGSTE experiments and appears in analyses of normal diffusion, anomalous diffusion, finite gradient pulse width effects, modified-Bloch equations, and anisotropic anomalous diffusion (Lin, 2017). The same usage appears in related treatments based on instantaneous signal attenuation, modified Gaussian phase distribution, Mainardi-Luchko-Pagnini phase distributions, and fractional or fractal derivatives (Lin, 2017, Lin, 2016, Lin, 2016, Lin, 2017, Lin, 2017).

A separate but adjacent usage appears in attosecond metrology. The paper on all-optical frequency resolved optical gating for isolated attosecond pulse reconstruction states that it does not use the phrase “Peripheral Frequency Gating” explicitly, but that the implemented concept is an all-optical frequency gating mechanism in which a weak synchronized field perturbs high-harmonic generation and acts as a phase gate (Yang et al., 2019). In that setting, the measured perturbed spectrum is formulated as a convolution of the unperturbed dipole and a phase gate, enabling conventional FROG retrieval (Yang et al., 2019).

The principal misconception to avoid, therefore, is terminological rather than technical. In the deep-learning setting represented by PFGNet, Peripheral Frequency Gating denotes a pixel-wise, frequency-guided peripheral gating block for adaptive convolutional filtering. In NMR/MRI, PFG ordinarily denotes pulsed field gradient. In attosecond pulse metrology, the relevant phrase is frequency resolved optical gating with a weak-field-induced phase gate. The shared acronym does not imply a shared formalism.