CTIM: Cross-Temporal Interaction Module

• CTIM is a structured neural module that fuses features across temporal and modality streams, enhancing tasks like semantic segmentation and action recognition.
• It employs techniques such as bidirectional state-space scanning, cross-attention, and diffusion-based matching to achieve efficient, linear-time temporal fusion.
• Empirical studies show significant performance improvements, with boosted mIoU, mAP, and recall, confirming CTIM’s versatility in various high-impact applications.

A Cross-Temporal Interaction Module (CTIM) is a structured neural subnetwork for modeling dependencies and feature fusion across temporal axes in multi-modal or multi-stream architectures. CTIMs operationalize fine-grained cross-time reasoning, cross-sequence modulation, and intention-context disentanglement, extending traditional spatial or intra-stream attention mechanisms. Employments of CTIM span semantic segmentation (image + event fusion), action recognition/anticipation, recommendation (user–item matching), and generative motion synthesis. Architectures and formulation details differ by domain, but all CTIMs undertake interleaving, cross-attending, and/or parameterizing features sampled from distinct temporal contexts or modalities.

1. Architectural Paradigms and Domain Placement

CTIMs are deployed in several high-impact architectures:

• Semantic Segmentation Fusion (MambaSeg): Dual-branch pipelines model RGB frames and event-voxel grids independently via state-space Mamba blocks; CTIM operates atop each scale’s outputs to synchronize modalities temporally, correcting misalignments between visual and event cues (Gu et al., 30 Dec 2025).
• Action Dynamics Reasoning (State-Specific Model, SSM): CTIM manages dynamic cross-talk between compressed historical states, present frame, and intention features, explicitly refining both action detection and anticipation heads (Yang et al., 12 Oct 2025).
• User–Item Matching (Diffusion Two-Tower): CTIM fuses diffusion-based intent prediction, session-level self-attention, and history-guided scoring inside a user tower, yielding rich cross-temporal user–item interactions (Wang et al., 28 Feb 2025).
• Human–Human Motion Synthesis (InterMamba): CTIM (coin: “Cross-ASTM”) enables joint spatio-temporal reasoning between two individuals’ motion histories, parameterizing SSM kernels on the coupled partner features and yielding high-fidelity dyadic motion (Wu et al., 3 Jun 2025).

This cross-cutting deployment demonstrates the generic utility of CTIM as a versatile vehicle for temporal and cross-modal reasoning.

2. Fundamental Computations and Module Structure

All CTIMs instantiate multi-phase operations with temporal fusion and attention:

• Temporal Interleaving & Weighting (MambaSeg):
  • Alternating insertion of event and image feature bins forms a $2T\times H\times W$ tensor, capturing joint time steps.
  • Attention weights $W^T_F$ are computed as

  $$W^T_F = \sigma(\mathrm{Conv}(F^T_{\max}) + \mathrm{Conv}(F^T_{\mathrm{avg}}))$$

  and are subsequently broadcast across the modalities, emphasizing motion-salient frames (Gu et al., 30 Dec 2025).

• Bidirectional State-Space Scan (MambaSeg):

  • S6 blocks are applied forward and backward over flattened spatial features, enabling bidirectional, long-range temporal context refinement at linear complexity.
  • Residual updates are gated via modality-aware temporal attention blocks.
• Cross-Attention (SSM):
  • For current-present refinement:

  $$F_c' = \mathrm{softmax}\left(\frac{Q_c K_t^\top}{\sqrt{d_k}}\right) V_t$$

  with $Q_c, K_t, V_t$ projected via learned matrices from historical, present, and intention features. - Future refinement similarly attends over the updated context, propagating mutual influence (Yang et al., 12 Oct 2025).

• Mixed-Attention + Diffusion (Two-Tower Matching):

  • Self-attention is run over the session plus generated intent embedding; history gating is computed via a query-key FFN and temporal lag embedding (Wang et al., 28 Feb 2025).
• Cross-SSM Kernel Parameterization (InterMamba):
  • Two parallel SSM branches for temporal and spatial axes, with partner features injected into kernel parameter generation via $h_{\mathrm{inter}}$.
  • Outputs from both branches (temporal, spatial) layer-normalized, fused by learnable scalars, and updated by gating functions.

A shared theme is the multi-stage pipeline: cross-temporal aggregation, attention modulated by context, and an explicit update via learnable gating or residual paths.

3. Mathematical Formulations and Pseudocode

Representative formulations summarize CTIM technical depth:

CTIM Variant	Key Mathematics	Principal Operations
MambaSeg	$F^T_i = \mathrm{Insert}(E_i,I_i)$, $W^T_F$, S6 forward/back	Interleaving, S6 scan, mask
SSM (Action)	$F_c' = \text{Att}(Q_c,K_t,V_t)$, $F_a' = \text{Att}(Q_a,K_{t}',V_{t}')$	Sequential cross-attention
Two-Tower Matching	Diffusion equations, $e_u = FFN([h_s; h_l])$	Diffusion, Transformer, gating
InterMamba	SSM kernel parametrization via partner features	Mix-SSM, gating, fusion

Complete pseudocode is provided in each source, enabling stepwise technical reconnaissance (Gu et al., 30 Dec 2025, Yang et al., 12 Oct 2025, Wang et al., 28 Feb 2025, Wu et al., 3 Jun 2025).

4. Quantitative Impact, Ablations, and Complexity

CTIM variants consistently demonstrate significant empirical improvements:

• MambaSeg: Addition of CTIM boosts mIoU from 74.38% (baseline) to 76.20% (CTIM only) and 77.56% (full DDIM). Component removal illustrates additive contributions from temporal attention (CTA), bi-directional scan (BTSS), and modality-aware update (TA) (Gu et al., 30 Dec 2025).
• SSM (Action): Full CTIM increases detection mAP from 46.1 to 71.8 and anticipation mAP from 43.9 to 58.1 on THUMOS’14. Ablations show interactions between past, present, and intention are synergistic (Yang et al., 12 Oct 2025).
• Diffusion Two-Tower: Integrated CTIM yields +11.8% relative Recall@2 and +22.3% MRR@20 on ML-1M; +10.98% online effective view rate and +37.4% average play time, all without notable runtime penalty (Wang et al., 28 Feb 2025).
• InterMamba: Cross-ASTM boosts R-Prec from 0.409 (self-ASTM only) to 0.605 (with cross-ASTM) and further to 0.705 (full block incl. LIIA), while halving parameter/FLOP counts and reducing inference time to 0.567 s/sample (Wu et al., 3 Jun 2025).

These studies establish that CTIM achieves linear-time temporal fusion with negligible parameter cost relative to quadratic-complexity Transformer-style attention.

5. Comparative Analysis: Mechanisms and Distinctions

Distinct CTIM implementations target complementary objectives:

• Modality Fusion (MambaSeg): CTIM specializes in aligning dynamic cues from sensor streams, preventing temporal misalignments and ambiguity, with an explicit focus on temporal intervals (Gu et al., 30 Dec 2025).
• Multistep Cross-Attention (SSM): CTIM encodes “mutual influence” between discrete temporal slices, modeling both current and intention states against historic context; two-step attention sequence ensures bidirectional dependency propagation (Yang et al., 12 Oct 2025).
• Diffusion-Driven Matching (Two-Tower): CTIM operationalizes session-intent history fusion in user representation learning, with explicit attention over temporal lags and adaptive drift; integrating diffusion-based next-intent generation adds anticipatory semantics (Wang et al., 28 Feb 2025).
• Cross-Sequence Modulation (InterMamba): CTIM generalizes state-space modeling to scenario-level inter-actor reasoning, parameterizing Mamba kernels dynamically with partner’s motion, achieving real-time joint synthesis and improved physical plausibility of outputs (Wu et al., 3 Jun 2025).

The core innovation of CTIM is in extending temporal or spatio-temporal fusion beyond simple self-attention or static pooling, enabling explicit modeling of inter-contextual dynamics.

6. Computational Complexity and Efficiency Considerations

All cited CTIMs are designed for linear complexity:

• Parameterization: Typically $O(T^2)$ (MambaSeg: $\sim$300 new parameters at $T=10$), $O(D\times D)$ for attention matrices (SSM, Two-Tower, InterMamba).
• FLOP Counts: CTIM S6 scans: $O(4T HW)$ (MambaSeg); InterMamba achieves near-linear $O(BLD)$; compared to $O(N^2)$ for Transformer cross-attention.
• Inference Efficiency: CTIM consistently yields sub-millisecond to sub-second inference costs at scale (e.g., 0.68 ms/user on Tesla T4; 0.567 s/sample on InterMamba).
• Parameter Sharing: MambaSeg and InterMamba re-use backbone state-space parameters, minimizing redundancy.

A plausible implication is that CTIM design principles may be generically applied to future scalable cross-modal architectures with minimal overhead.

7. Application-Specific Role and Future Directions

CTIM modules currently underpin advances in:

• Multi-modal fusion (image-event, video-intention-action, user–item intent modeling)
• Robust dynamic segmentation (temporal flicker correction, motion stability)
• Anticipative and interactive generation (joint motion synthesis, text-action alignment)
• Scalable user modeling in industrial recommender systems

Continued expansion into domains requiring cross-context and temporal alignment—multimodal dialogue, reinforcement learning with latent state anticipation, dyadic interaction modeling—is anticipated. Formalization of CTIM as a generic cross-temporal attention template may lead to standardized practices for complex temporal reasoning under computational constraints.