HyperTea: High-Order Temporal Modeling in MIRSTD

• HyperTea is a hypergraph-based temporal enhancement and alignment network designed to detect small, low-intensity targets by modeling high-order spatiotemporal correlations.
• It employs a multi-scale architecture with a global temporal enhancement module (GTEM), local temporal enhancement module (LTEM), and temporal alignment module (TAM) to capture both long-range context and adjacent-frame dynamics.
• Experiments on DAUB and IRDST report approximately +4% [email protected] and +2% F1-score improvements over previous methods, underlining its practical advantages in challenging MIRSTD scenarios.

HyperTea is a hypergraph-based temporal enhancement and alignment network for moving infrared small target detection (MIRSTD). It is designed for scenarios in which targets are small, weak in intensity, and subject to complex motion patterns, and it addresses two limitations identified in prior MIRSTD pipelines: the predominance of low-order correlation modeling and the tendency to perform feature extraction and enhancement within a single temporal scale. HyperTea integrates global and local temporal perspectives to model high-order spatiotemporal correlations of features, and it consists of a global temporal enhancement module (GTEM), a local temporal enhancement module (LTEM), and a temporal alignment module (TAM). The method is described as the first work to integrate convolutional neural networks (CNNs), recurrent neural networks (RNNs), and hypergraph neural networks (HGNNs) for MIRSTD, and experiments on DAUB and IRDST report state-of-the-art performance (Qi et al., 14 Aug 2025).

1. Problem setting and motivation

Moving infrared small target detection concerns the identification of tiny, low-contrast, and low-SNR targets in infrared imagery across time. The problem is difficult because such targets lack texture, are readily camouflaged by background noise, and exhibit high spatial and temporal coupling with clutter. The motion dynamics are also described as complex and non-linear, with both intrinsic target motion and platform motion affecting representation quality (Qi et al., 14 Aug 2025).

Within this setting, conventional architectural choices exhibit distinct limitations. CNNs are effective for spatial feature extraction but limited in non-local modeling. RNNs, including ConvLSTM-style formulations, can exploit temporal dependencies but are limited to low-order relationships and are inefficient for long sequences. Transformers can model fully connected dependencies, but the reported drawbacks include quadratic cost and loss of spatial detail induced by patch embedding. HyperTea is positioned against this background as a method that targets high-order relationships, specifically many-to-many dependencies among features, which the paper argues are important for separating targets from clutter when spatial and temporal patterns overlap (Qi et al., 14 Aug 2025).

A central motivation is the use of hypergraphs. In a hypergraph, one hyperedge can connect multiple nodes, allowing feature groups rather than only pairs to participate in message passing. In the HyperTea formulation, this high-order structure is used to capture both globally aggregated temporal semantics and locally evolving motion patterns. This suggests a shift from pairwise spatiotemporal aggregation toward explicit high-order spatiotemporal reasoning in MIRSTD.

2. Overall architecture and data flow

HyperTea is organized as a unified multi-timescale, multi-order model with a CSPDarknet-based CNN backbone and three principal modules: GTEM, LTEM, and TAM (Qi et al., 14 Aug 2025). The input is a sequence of $T$ frames,

$$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$

and the output is the detection result for the frame $\mathbf{I}_T$.

The shared backbone first produces spatial features

$$\mathbf{F}_s = \{ \mathbf{F}_1, \ldots, \mathbf{F}_T \} \in \mathbb{R}^{T \times C \times H \times W}.$$

These features are then processed along two temporal pathways. The global pathway aggregates all frames jointly and uses hypergraph convolution to inject high-order semantic context back into the per-frame representations. The local pathway uses a hypergraph-enhanced recurrent cell to model adjacent-frame motion patterns. Finally, TAM aligns and fuses the global and local streams so that multi-timescale information can be combined without cross-scale feature mismatch (Qi et al., 14 Aug 2025).

The architecture therefore separates temporal modeling into two complementary regimes. GTEM addresses long-range or sequence-level context; LTEM focuses on short-range temporal evolution between adjacent frames; TAM reconciles the two. The paper’s ablation results report that all three modules are necessary for top performance, which indicates that high-order reasoning, short-range temporal dynamics, and alignment are treated as distinct but interdependent design requirements (Qi et al., 14 Aug 2025).

3. Global temporal enhancement module

GTEM is designed to provide each frame with globally contextual, high-order semantic features spanning all input frames. It begins with temporal aggregation by concatenating frame features and applying convolution:

$$\mathbf{L}_{gt} = \mathrm{Conv}_{3\times3}^2 \left( \mathrm{Concat}\left[\mathbf{F}_1, ..., \mathbf{F}_T\right] \right).$$

A direction-preferred convolution block (DPCB) is then applied to extract features with different directional preferences. The stated purpose is to discriminate stripy or linear background artifacts. The block is defined as

$$\begin{aligned} \mathbf{L}_h &= \mathrm{DWC}(\mathrm{Conv}_{1\times5}(\mathbf{L}_{gt})), \ \mathbf{L}_v &= \mathrm{DWC}(\mathrm{Conv}_{5\times1}(\mathbf{L}_{gt})), \ \mathbf{L}_s &= \mathrm{DWC}(\mathrm{Conv}_{1\times1}(\mathbf{L}_{gt})), \ \mathbf{D}_{gt} &= \mathrm{Conv}_{3\times3}\left( \mathrm{Concat}[\mathbf{L}_h, \mathbf{L}_v, \mathbf{L}_s] \right) + \mathbf{L}_{gt}. \end{aligned}$$

Here, DWC denotes depth-wise convolution (Qi et al., 14 Aug 2025).

The resulting feature tensor is processed by a hypergraph convolution unit (HCU):

$$\mathrm{HCU}(\mathbf{X}) = \mathbf{X} + \mathbf{D}_v^{-1} \mathbf{H} \mathbf{D}_e^{-1} \mathbf{H}^T \mathbf{X} \Theta.$$

In this expression, $\mathbf{H}$ is the incidence matrix, $\mathbf{D}_v$ and $\mathbf{D}_e$ are the degree matrices of vertices and hyperedges, and $$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$0 denotes learned weights. The operation propagates messages among nodes and hyperedges so that many-to-many semantic dependencies can be represented. Applied to GTEM, it yields

$$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$1

The final GTEM step propagates the resulting global semantic summary back to each frame:

$$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$2

This produces the globally enhanced temporal feature set $$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$3 (Qi et al., 14 Aug 2025). In operational terms, GTEM converts sequence-wide temporal aggregation into per-frame features enriched by high-order semantic context.

4. Local temporal enhancement and temporal alignment

LTEM is intended to capture local spatiotemporal motion patterns, especially between adjacent frames, while maintaining high-order interactions. Its core computational unit is the Hypergraph Convolution LSTM, or HCCell, which augments a ConvLSTM-style recurrent update with hypergraph convolution:

$$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$4

Here, $$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$5 is the current input, $$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$6 is the previous hidden state, $$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$7 is the cell state, and $$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$8 is elementwise multiplication. According to the paper, this structure allows local temporal context to be modeled with high-order correlations rather than only pairwise recurrent dependencies (Qi et al., 14 Aug 2025).

The third major component, TAM, addresses misalignment between global and local temporal features. The module first normalizes and patch-embeds the two streams:

$$\mathbf{I}_s = [\mathbf{I}_1, \ldots, \mathbf{I}_T],$$9

It then constructs queries from local features and keys and values from global features:

$\mathbf{I}_T$0

The resulting global-local temporal attention is

$\mathbf{I}_T$1

followed by the residual fusion

$\mathbf{I}_T$2

TAM concludes with a Channel Spatial Aggregation Module (CSAM). The reported operations include a spatial attention map

$\mathbf{I}_T$3

a channel-importance estimate

$\mathbf{I}_T$4

and the fusion

$\mathbf{I}_T$5

The final fused representation $\mathbf{I}_T$6 is then sent to the detection head (Qi et al., 14 Aug 2025). The empirical interpretation given in the paper is that TAM is critical for avoiding local/global feature misalignment.

5. Optimization objective and empirical performance

HyperTea is trained with standard detection losses:

$\mathbf{I}_T$7

where the terms correspond to bounding box regression, classification, and object probability, respectively (Qi et al., 14 Aug 2025).

The reported evaluation datasets are DAUB and IRDST. DAUB is described as containing dim-small aircraft under varied but stable backgrounds, whereas IRDST is described as a larger and more temporally complex dataset for infrared small targets. The evaluation metrics listed are mAP@0.5, Precision, Recall, F1-score, and FPS/inference complexity (Qi et al., 14 Aug 2025).

Dataset	HyperTea	Previous SOTA
DAUB	[email protected] 95.59; F1 98.23	91.52; 96.17 (Tridos)
IRDST	[email protected] 76.42; F1 87.62	72.54; 85.63 (Tridos)

On DAUB, HyperTea reports [email protected] of 95.59 and F1 of 98.23, compared with 91.52 and 96.17 for the previous SOTA. On IRDST, it reports [email protected] of 76.42 and F1 of 87.62, compared with 72.54 and 85.63 for the previous SOTA. The paper summarizes these margins as approximately $\mathbf{I}_T$8 [email protected] and $\mathbf{I}_T$9 F1 on both datasets (Qi et al., 14 Aug 2025).

The ablation studies further report that GTEM, LTEM, and TAM are all necessary for the best results; that hypergraph-based high-order reasoning in both GTEM and LTEM yields substantial gains over CNN- or RNN-only baselines; and that the attention-based alignment in TAM is critical. In terms of deployment-oriented characteristics, HyperTea is reported to maintain 14.5 FPS and 13.75M parameters, and to outperform Hyper-YOLO and other hypergraph-detection hybrids, which the paper interprets as evidence that temporal multi-scale modeling and feature alignment are essential rather than optional additions (Qi et al., 14 Aug 2025).

6. Technical significance and scope

The principal technical contribution of HyperTea lies in combining three modeling paradigms within a single MIRSTD pipeline: CNNs for spatial representation, RNNs for short-term temporal evolution, and HGNNs for high-order correlation learning. The architecture does not treat these components as parallel embellishments; instead, it assigns each a different role in a staged temporal reasoning process. GTEM models sequence-level global temporal context through semantic aggregation and hypergraph propagation. LTEM models adjacent-frame temporal dynamics through recurrent updates enriched by hypergraph convolution. TAM then aligns the two feature streams so that global semantics and local motion evidence can be fused coherently (Qi et al., 14 Aug 2025).

A second contribution is the explicit multi-timescale design. Existing methods are characterized in the paper as typically performing feature extraction and enhancement within a single temporal scale. HyperTea instead distinguishes between global and local temporal context and introduces a dedicated module to reconcile them. This separation is significant because, in the problem formulation adopted by the paper, background evolution and target motion need not manifest on the same timescale. A plausible implication is that the method’s reported gains derive not only from high-order correlation learning, but also from the decision to represent long-range context and adjacent-frame motion with different operators.

The scope of the reported evidence is defined by DAUB and IRDST, the listed metrics, and the ablation analyses. Within that scope, HyperTea is presented as a state-of-the-art MIRSTD method and as the first model in this domain to integrate CNNs, RNNs, and HGNNs. Its source code is reported as publicly available at the project repository associated with the paper (Qi et al., 14 Aug 2025).