Papers
Topics
Authors
Recent
Search
2000 character limit reached

Cross Spatial Temporal Fusion (CSTF)

Updated 7 July 2026
  • Cross Spatial Temporal Fusion (CSTF) is a design principle that jointly models spatial structures and temporal contexts to prevent representational losses.
  • It leverages diverse computational motifs—such as temporal gating, cross-attention, and state space models—to enable adaptive and long-range fusion.
  • CSTF has demonstrated empirical gains across domains like change detection, autonomous driving, and multimodal tracking by merging insights from spatial and temporal signals.

Searching arXiv for papers that explicitly use or closely align with Cross Spatial Temporal Fusion / cross spatio-temporal fusion terminology. Cross Spatial Temporal Fusion (CSTF) denotes a family of model designs that jointly integrate spatial structure and temporal context, typically to avoid the representational losses that arise when the two are processed in isolation. In the surveyed literature, CSTF appears in remote sensing change detection, bird’s-eye-view perception, collaborative perception, deepfake detection, RGB-T tracking, pedestrian detection, traffic forecasting, and cross-modal image matching. The common pattern is not a single standardized operator but a coupled fusion strategy in which temporal cues modulate spatial selection, spatial cues guide temporal aggregation, or both are embedded in one unified formulation (Qin et al., 2022, Ma et al., 2023).

1. Emergence and conceptual scope

A major motivation for CSTF is the observation that many earlier systems learned spatial and temporal dependencies separately and merged them only afterward. In autonomous driving, UniFusion proposed “a new method that unifies both spatial and temporal fusion and merges them into a unified mathematical formulation,” arguing that the resulting unified fusion can support long-range fusion, make temporal fusion adaptive with learnable weights, and avoid information loss in conventional BEV fusion methods (Qin et al., 2022). In traffic forecasting, DSTCGCN explicitly criticized methods that “learn spatial and temporal dependencies separately,” because such separation ignores dependencies that cross spatial and temporal dimensions (Wu et al., 2023). In deepfake detection, CAST similarly argued that CNN-Transformer hybrids often process spatial and temporal features independently and then combine them with averaging, addition, or concatenation, which limits the depth of spatio-temporal interaction (Thakre et al., 26 Jun 2025).

The literature also shows that CSTF is broader than modern attention-only formulations. An earlier antecedent is TFusion for unsupervised cross-dataset person re-identification, which fused visual similarity with learned spatio-temporal patterns through a Bayesian fusion model and then refined the visual model through a learning-to-rank mutual promotion procedure (Lv et al., 2018). This suggests that the core CSTF idea predates current transformer and state-space implementations: the essential objective is coordinated inference over spatial and temporal evidence, not adherence to one particular neural primitive.

A second conceptual distinction concerns what “temporal” means. In RSCD, STNet operates on two co-registered images T1T_1 and T2T_2, so temporal fusion is bi-temporal rather than video-based (Ma et al., 2023). In CAST and MambaST, temporal fusion is performed over frame sequences (Thakre et al., 26 Jun 2025, Gao et al., 2024). In CollaMamba and BEVFusion4D, it is historical BEV aggregation across driving frames (Li et al., 2024, Cai et al., 2023). In DSTCGCN, it is a dependency graph over selected traffic time steps (Wu et al., 2023). CSTF is therefore best understood as a cross-domain design principle for jointly modeling spatial and temporal dependencies, rather than as a task-specific block.

2. Core computational motifs

One canonical CSTF pattern is explicit temporal gating followed by spatial refinement. STNet implements this with a Temporal Feature Fusion (TFF) module and a Spatial Feature Fusion (SFF) module. At each scale, TFF first computes a coarse change map, then derives dual context encodings, generates cross-temporal gates, and produces a change-emphasized feature: ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),

W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),

Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).

SFF then fuses coarse and fine features by cross-scale attention so that high-level semantics guide low-level spatial detail recovery (Ma et al., 2023).

A second motif is token-level cross-attention between temporal and spatial streams. CAST extracts spatial tokens from CNN feature maps and temporal tokens from globally pooled frame descriptors, encodes the temporal sequence with a Transformer encoder, averages spatial tokens across time, and lets the temporal sequence query the pooled spatial tokens: Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,

A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.

The fused representation is then formed by residual addition and layer normalization, followed by a feed-forward block (Thakre et al., 26 Jun 2025). CFBT applies a related logic inside RGB-T tracking, but distributes fusion over three modules: Cross Spatio-Temporal Augmentation Fusion (CSTAF), Cross Spatio-Temporal Complementarity Fusion (CSTCF), and Dual-Stream Spatio-Temporal Adapter (DSTA), all inserted into selected transformer layers (Zeng et al., 2024).

A third motif is long-range linear-complexity modeling through state space models. CollaMamba builds a cross-agent spatial-temporal collaborative state space model in which Mamba2D scans capture long-range spatial causal dependencies and a history-aware temporal SSM boosts current features using extended historical frames. Its continuous SSM is written as

ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,

with a discretized recurrent form

ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.

Cross-agent fusion is then performed by Mamba Fusion blocks over concatenated agent sequences, while a temporal trajectory encoder refines features from historical observations (Li et al., 2024). MambaST uses a comparable SSM principle for RGB-thermal pedestrian detection, but combines it with Multi-head Hierarchical Patching and Aggregation (MHHPA) to preserve fine and coarse spatial detail across scales (Gao et al., 2024).

A fourth motif is explicit graph construction over cross-space-time dependencies. DSTCGCN first uses an FFT-based attentive selector to choose relevant time steps, then constructs a dynamic spatial graph AtSA_t^S, a set of temporal connection graphs T2T_20, and fuses them into a block-triangular cross graph T2T_21. This unified graph is then processed by graph convolution, so one sweep spans both spatial and selected non-local temporal dependencies (Wu et al., 2023).

A fifth motif is deformable or geometry-aware alignment in BEV. BEVFusion4D first uses the LiDAR-Guided View Transformer (LGVT) to synthesize camera BEV under LiDAR spatial guidance, then performs Temporal Deformable Alignment (TDA) over historical BEV features after ego-motion calibration, producing a final fused BEV feature for 3D detection (Cai et al., 2023).

3. Representative architectural families

In remote sensing change detection, STNet exemplifies a tightly coupled CSTF pipeline with a Siamese ResNet-18 encoder, per-scale TFF modules, progressive SFF from coarse to fine scales, and a decoder that concatenates all fused maps, applies a Channel-Attention Module, and predicts the final binary change mask (Ma et al., 2023). The stated purpose is to emphasize “real” changes while suppressing distractors such as illumination variation or vegetation cycles, then recover fine-grained spatial details through cross-scale attention.

In BEV perception, UniFusion frames the problem more abstractly by unifying spatial and temporal fusion in one mathematical formulation, with long-range fusion and learnable temporal weights as explicit capabilities (Qin et al., 2022). BEVFusion4D instantiates a more concrete multimodal CSTF pipeline: LiDAR features form T2T_22, camera features are lifted into BEV by LGVT to form T2T_23, the two are spatially fused by convolution, and TDA aggregates fused BEV features across T2T_24 frames (Cai et al., 2023). CollaMamba extends the idea to multi-agent collaborative perception, where the spatial dimension includes both intra-agent and cross-agent dependencies and the temporal dimension includes history-aware feature boosting as well as a collaborative prediction mode for missing messages (Li et al., 2024).

In video and multimodal analysis, CAST uses temporal-to-spatial cross-attention for deepfake detection so that temporal tokens can dynamically attend to spatial regions associated with subtle artifacts such as flickering eyes or warped lips (Thakre et al., 26 Jun 2025). MambaST targets RGB-thermal pedestrian detection and treats cross-spectral and temporal fusion jointly through patch-level SSM processing across multiple patch scales (Gao et al., 2024). CFBT uses cross spatio-temporal fusion in RGB-T tracking, where template augmentation, search-branch complementarity, and adapter-based fusion are distributed over selected transformer layers and add less than T2T_25 of total modal parameters (Zeng et al., 2024).

In remote sensing object detection via image feature matching, the 2025 CSTF model uses a U-shaped FCN with a CSTF block inserted between encoder and decoder. Its Patch Partition & Embedding acts as a scale-adaptive keypoint mechanism, Channel Cross-Attention and Spatial Cross-Attention generate soft correspondence maps, and a small FCN with SoftMax reformulates patch-pair similarity as a two-class classification problem (Amit et al., 25 Jul 2025). The same paper explicitly presents CSTF as a bridge between local distinctive keypoints and broader contextual information.

In traffic forecasting, DSTCGCN applies CSTF in a graph-theoretic form rather than through image tokens. Its fusion is built from a dynamic cross graph rather than from cross-attention, underscoring that CSTF is a modeling strategy, not a fixed implementation template (Wu et al., 2023).

4. Supervision, losses, and optimization regimes

The supervised objective in CSTF systems is typically inherited from the downstream task rather than from a universal fusion loss. STNet uses a hybrid of focal loss and dice loss to address extreme class imbalance in change detection: T2T_26 with T2T_27 and T2T_28. Its reported optimization settings are Adam, T2T_29, weight decay ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),0, multi-step learning-rate decay with ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),1, batch size ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),2, and random horizontal/vertical flips with ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),3 rotations (Ma et al., 2023).

CAST uses binary cross-entropy with logits for deepfake detection, together with Adam, ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),4, weight decay ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),5, batch size ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),6 video clips of ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),7 frames, and dropout ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),8 in attention and FFN blocks (Thakre et al., 26 Jun 2025). CFBT uses a tracking objective

ΔR=R1R2,Rc1=ψ([R1,ΔR]),Rc2=ψ([R2,ΔR]),\Delta R = R_{1}-R_{2}, \quad R_{c1}=\psi\bigl([R_{1},\Delta R]\bigr), \quad R_{c2}=\psi\bigl([R_{2},\Delta R]\bigr),9

where W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),0 is weighted focal loss, W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),1 is generalized IoU loss, and W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),2 is W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),3 regression loss; it is trained on LasHeR with batch size W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),4, W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),5 epochs, AdamW, weight decay W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),6, and an initial learning rate of W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),7 decayed by W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),8 after W1=σ(ϕ(Rc1)),W2=σ(ϕ(Rc2)),W_{1}=\sigma\bigl(\phi(R_{c1})\bigr), \quad W_{2}=\sigma\bigl(\phi(R_{c2})\bigr),9 epochs (Zeng et al., 2024).

The 2025 remote sensing CSTF model uses cross-entropy over match/non-match logits derived from attention or correlation maps: Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).0 Its reported optimizer is Adam with Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).1, initial learning rate Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).2, weight decay Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).3, training for Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).4 epochs, a learning-rate decay of Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).5 at epochs Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).6 and Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).7, and warm-up for the first Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).8 iterations (Amit et al., 25 Jul 2025).

Graph-based CSTF again follows task-native supervision. DSTCGCN maps the final hidden state to an Rt=ψ([W1R1,  W2R2]).R_{t}=\psi\Bigl([\,W_{1}\odot R_{1},\;W_{2}\odot R_{2}\,]\Bigr).9-step forecast and minimizes an Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,0 loss over the prediction horizon (Wu et al., 2023). The surveyed literature therefore suggests that CSTF is usually optimized end-to-end through the task head; the fusion module itself is rarely supervised in isolation.

5. Empirical performance and efficiency

The literature reports consistent gains when cross spatial-temporal coupling replaces separable fusion or naïve aggregation. The following examples are representative.

System Task Reported result
STNet (Ma et al., 2023) RSCD F1 Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,1 on WHU, Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,2 on LEVIR-CD, Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,3 on CLCD
BEVFusion4D (Cai et al., 2023) 3D object detection Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,4 mAP and Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,5 NDS on nuScenes val; Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,6 mAP and Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,7 NDS on test
CollaMamba-ST (Li et al., 2024) Collaborative 3D detection OPV2V [email protected] / [email protected] Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,8
CAST-B5 / CAST-B0 (Thakre et al., 26 Jun 2025) Deepfake detection Intra-dataset AUC Q=ZWQ,K=SmeanWK,V=SmeanWV,Q = \mathcal Z\,W^Q,\quad K = S_{\mathrm{mean}}\,W^K,\quad V = S_{\mathrm{mean}}\,W^V,9, accuracy A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.0; cross-dataset AUC A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.1 on unseen DeepfakeDetection
MambaST (Gao et al., 2024) RGB-thermal pedestrian detection LAMR A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.2, Recall A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.3
CFBT (Zeng et al., 2024) RGB-T tracking LasHeR PR / NPR / SR A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.4
CSTF remote sensing model (Amit et al., 25 Jul 2025) Object detection via feature matching Average mAP A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.5 on HRSC2016 and A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.6 on DOTA; inference speed A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.7 FPS

Ablation studies are especially informative. In STNet, the base model without TFF or SFF yields F1 A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.8, Base + TFF only yields A=softmax ⁣(QKTdk),Z^=AV.A = \mathrm{softmax}\!\Bigl(\tfrac{QK^T}{\sqrt{d_k}}\Bigr),\quad \widehat{\mathcal Z}=A\,V.9, Base + SFF only yields ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,0, and full STNet yields ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,1; the supplied summary states that TFF alone brings ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,2, SFF alone ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,3, and combining them ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,4 (Ma et al., 2023). In BEVFusion4D, BEVFusion4D-S reports ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,5 mAP / ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,6 NDS on nuScenes val, while adding TDA raises performance to ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,7 mAP / ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,8 NDS; naïve concatenation of the last five BEV frames gives only ht=Aht+Bxt,ot=Cht+Dxt,h'_t = A\,h_t + B\,x_t,\quad o_t = C\,h_t + D\,x_t,9 mAP (Cai et al., 2023). In CollaMamba, “Simple” versus “ST” shows that adding the history-aware temporal SSM gives ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.0 [email protected] on OPV2V, while “ST” versus “Miss” shows that collaborative prediction trades a small accuracy drop of less than ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.1 [email protected] for robustness under poor communication (Li et al., 2024).

Efficiency is highly architecture-dependent. STNet reports only ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.2M parameters and ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.3G FLOPs per ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.4 input, described as the lowest compute cost among leading methods (Ma et al., 2023). MambaST reports ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.5 GFLOPs and ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.6M parameters, versus ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.7 GFLOPs and ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.8M for Deformable CFT, while also improving LAMR (Gao et al., 2024). CollaMamba reports reductions in computational and communication overhead by up to ht=Aˉht1+Bˉxt,ot=Cht+Dxt.h_t = \bar A\,h_{t-1} + \bar B\,x_t,\quad o_t = C\,h_t + D\,x_t.9 and AtSA_t^S0, respectively (Li et al., 2024). By contrast, the remote sensing CSTF attention model reports AtSA_t^S1M parameters, AtSA_t^S2G FLOPs, and AtSA_t^S3 FPS on a single NVIDIA V100, and also notes increased compute and latency from dual attention paths and the FCN classifier (Amit et al., 25 Jul 2025).

6. Interpretation, misconceptions, and open directions

The surveyed literature suggests several recurrent misconceptions. First, CSTF is not synonymous with attention. Attention-based fusion is common, but STNet relies on cross-temporal gating and cross-scale attention, DSTCGCN relies on dynamic cross graphs, CollaMamba and MambaST rely on state space models, and TFusion uses Bayesian fusion of visual and spatio-temporal cues (Ma et al., 2023, Wu et al., 2023, Li et al., 2024, Lv et al., 2018). Second, CSTF is not restricted to multimodal settings. It appears in bi-temporal change detection from co-registered images, traffic forecasting from time-varying sensor graphs, and monocular or dual-template tracking, as well as in multimodal RGB-thermal and LiDAR-camera systems (Ma et al., 2023, Wu et al., 2023, Zeng et al., 2024, Cai et al., 2023). Third, temporal fusion is not always long-horizon video modeling; it may refer to two time points, a selected set of historical steps, or a short BEV window.

Open research directions are stated most explicitly in the 2025 remote sensing CSTF work: learnable dynamic patch sizing, unsupervised or self-supervised domain adaptation for new modalities, extension to video or continuous-time matching by adding a temporal encoder and enforcing consistency across multiple frames, application to cross-modal 3D-2D matching, and incorporation of geometric constraints such as epipolar or homography priors into attention maps as soft regularizers (Amit et al., 25 Jul 2025). More broadly, the literature suggests that future CSTF research will continue to balance three competing demands: stronger cross-dimensional coupling, longer effective context, and lower computational or communication cost.

A plausible implication is that CSTF will remain a heterogeneous research area rather than converging on a single canonical block. The strongest common denominator is methodological: spatial and temporal evidence are made mutually informative within the fusion operator itself, instead of being combined only after separate encoding.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Cross Spatial Temporal Fusion (CSTF).