Frequency Decoupled Spatiotemporal Module

• The FDSCM is designed to decouple spatiotemporal signals in frequency space, reducing interference between global and local dynamics.
• It employs parallel processing paths with spectral transforms (e.g., FFT, Chebyshev) to enhance signal separability in applications like UAV video anomaly detection and radar nowcasting.
• Empirical studies show that integrating FDSCM improves performance metrics, with gains such as a 3.5% AUC increase in UAV anomaly detection and enhanced forecasting accuracy in radar and traffic prediction.

A Frequency Decoupled Spatiotemporal Correlation Module (FDSCM) is a modular block designed to explicitly disentangle and model distinct facets of spatiotemporal dependencies—often within high-dimensional video or sequence data—by leveraging spectral analysis and adaptive, parallel correlation mechanisms. FDSCMs have shown empirical benefit in tasks such as unmanned aerial vehicle (UAV) video anomaly detection, precipitation nowcasting from radar echoes, and heterogeneous-frequency traffic speed prediction, where multi-source dynamics and intricate spatiotemporal interactions cannot be efficiently represented by either pure spatial or pure temporal modeling alone. Implementations differ by domain, but always seek to orthogonalize or decouple spatiotemporal frequency components to reduce interference and selectively enhance meaningful signals (Liu et al., 16 Jan 2026, Xu et al., 2024, Sun et al., 2021).

1. Theoretical Motivation and Decoupling Principle

FDSCMs address the challenge of "multi-source motion coupling," where the observed temporal evolution in high-dimensional time-series (e.g., video frames, radar echoes) stems from an admixture of global and local processes. For UAV video anomaly detection, global motion (ego-motion from the platform) and local scene/object motion coexist, with their frequency signatures superposed in both space and time (Liu et al., 16 Jan 2026). In precipitation nowcasting, weather radar echoes exhibit complex correlations—some global and others localized or transient—making naive coupled modeling susceptible to mutual interference (Xu et al., 2024). For traffic prediction, road segments produce measurements at different frequencies, and both spatial graph structure and temporal periodicities (daily, weekly, etc.) co-propagate (Sun et al., 2021).

Frequency-domain decoupling leverages the observation that these sources manifest distinct spectral profiles: global/periodic background motion yields highly structured spectral energy (e.g., energy concentrated at central axes), while local motion or anomalous events result in broader, more isotropic spectral spreads. By projecting features into frequency space, FDSCMs can adaptively weight, filter, or orthogonalize these contributions, disentangling confounded signals that spatial or temporal-only models cannot separate.

2. Core Module Designs Across Applications

Despite task-specific adaptations, FDSCM architectures generally implement the following principles:

• Parallel Decoupled Paths: Features are processed by distinct branches, each targeting a specific aspect—global spatiotemporal correlation, local spatial structure, and temporally-evolving patterns with frequency enhancement.
• Spectral Analysis: Frequency transforms (discrete Fourier or Chebyshev polynomial expansions) are integral, either for temporal frequency analysis, spatial projection, or graph convolution.
• Attention and Fusion: Outputs from parallel branches are fused—by summation, residual combination, or attention-based weighting—yielding a representation with disentangled global and local dynamics.

Table 1 summarizes FDSCM architectures in three major domains:

Domain	Parallel Paths	Spectral Treatment
UAV Video VAD	Temporal frequency decoupling, spatiotemporal autocorrelation	FFT on time & space; no learnable params (Liu et al., 16 Jan 2026)
Radar Nowcasting	Spatiotemporal, spatial, temporal+frequency SFT-Blocks	Swin-transformer, learnable freq mixing (Xu et al., 2024)
Traffic Prediction	Heterogeneous graph, LSTM for multi-periodicity, attention fusion	Chebyshev polynomials on graph/time (Sun et al., 2021)

3. Mathematical Formulation: UAV Video Anomaly Detection

The FDSCM in FTDMamba operates as follows (Liu et al., 16 Jan 2026):

• Input: Feature tensor $$f \in \mathbb{R}^{B \times T \times C \times H \times W}$$ after multi-scale encoding.
• Temporal Frequency Decoupling: For each position, perform 1D FFT along time:

$$\hat f_k(b,c,h,w) = \sum_{t=0}^{T-1} f(b,t,c,h,w)\,e^{-j2\pi kt/T}$$

Adaptive frequency weighting:

$$w_k(b,c,h,w) = l_k^2 (A_k(b,c,h,w))^2,\quad A_k = |\hat f_k|$$

Weighted spectrum and inverse FFT reconstruct the cleaned temporal signal.

• Spatiotemporal Correlation: Collapse spatial grid to $S=H\,W$, then perform 2D FFT over $(t, s)$, compute power spectral density, and recover the correlation map via inverse FFT—real part yields autocorrelation attention:

$$R_{f'}^{(b,c)}(t,s) = \Re\Biggl\{\frac{1}{TS} \sum_{f_t,f_s} S_{f'}^{(b,c)}(f_t,f_s) e^{j2\pi(\frac{f_t t}{T}+\frac{f_s s}{S})}\Biggr\}$$

• Output: Enhanced feature

$\bar f = f' + R_{f'} \odot f'$

where $\odot$ denotes elementwise multiplication.

No learnable parameters are introduced in the FFT-based decoupling, and frequency weight computation is data-driven.

4. Architectural Integrations and Roles

FDSCM modules are typically positioned mid-network, interfacing with generic encoders/decoders:

• FTDMamba (UAV video):
  • Post-encoder pyramid-transformer features are fed in parallel to FDSCM and a separate Temporal Dilation Mamba Module (TDMM).
  • Outputs $\bar f$ (from FDSCM) and $\tilde f$ (from TDMM) are channel-concatenated, projected, and sent to the decoder (Liu et al., 16 Jan 2026).
• SFTformer (Radar echo):
  • Stacks of SFT-Blocks (each an FDSCM) provide deep hierarchical correlation decoupling, with each block's output the sum of its three decoupled paths.
  • The architecture supports joint prediction-reconstruction training, reconstructing odd/even frame splits and future echo predictions, improving model memory and forecasting (Xu et al., 2024).
• MCAN (Traffic):
  • FDSCM is conceptually realized as the combination of Heterogeneous Spatial Correlation (via spectral graph convolution and Chebyshev fill), Multi-fold Temporal Correlation (parallel LSTMs for different periodicities), and a context-aware attention-fusion layer (Sun et al., 2021).

5. Empirical Results and Ablation Studies

Extensive ablation and benchmarking evidence the FDSCM's benefit:

• UAV Video VAD: Introduction of FDSCM alone yields +1.9% (UIT-ADrone) / +3.5% (MUVAD) micro-AUC improvement over baseline. Removing either temporal frequency decoupling or spatiotemporal correlation reduces performance by 2–4 points, directly quantifying each subpath's criticality (Liu et al., 16 Jan 2026).
• Radar Echo Prediction: SFTformer outperforms strong convolutional and recurrent baselines by 3–5 points (CSI) and 4–6 points (HSS). Ablations reveal "no T" most degrades low-rainfall CSI, "no S" degrades morphological fidelity, and "no ST" weakens long-horizon accuracy (Xu et al., 2024).
• Traffic Speed Prediction: MCAN's attention-fused FDSCM achieves lower MSE than pure temporal or spatial baselines, with explicit modeling of frequency-heterogeneous measurements across road segments. All subcomponents (HSC, MTC, attention) are necessary for optimal accuracy, as evidenced by ablation losses (Sun et al., 2021).

6. Implementation and Training Considerations

• FFT-based Variants: Typically have no trainable parameters in the decoupling layer; frequency weights are computed per-feature location per mini-batch (Liu et al., 16 Jan 2026).
• Swin-Transformer/Attention-Based Blocks: Deploy standard window or shifted-window multihead attention, 1×1 convolutions, MLPs, and frequency mixing via learnable Fourier kernels or Chebyshev coefficients (Xu et al., 2024, Sun et al., 2021).
• Loss Functions: Shared global objectives combine intensity, gradient, and structural similarity losses (for video), or joint reconstruction and prediction losses (for sequence forecasting), with tunable weighting (Liu et al., 16 Jan 2026, Xu et al., 2024).

7. Distinctions and Comparative Advantages

FDSCMs fundamentally differ from prior spatiotemporal modules that conflate temporal and spatial dependencies in a joint latent space. By explicit architectural and spectral decoupling:

• Interference between spatial and temporal modeling is reduced, mitigating overfitting to global drift at the expense of object or event discrimination.
• Temporal frequency analysis injects periodicity priors and improves recall of rare or regular patterns (e.g., by mitigating RNN-forgetting in sequence models).
• Empirically, FDSCM designs present modularity—allowing integration with diverse backbone encoders—while consistently enhancing representation sharpness and anomaly salience, especially in dynamic, multi-source environments (Liu et al., 16 Jan 2026, Xu et al., 2024).

Overall, FDSCM provides a paradigmatic tool for applications where joint spatiotemporal modeling must robustly separate coupled, heterogeneous, or frequency-mismatched signals. Its principles and instantiations span video, weather forecasting, and traffic prediction, evidencing both generality and domain-specific adaptability.