---
title: 'SeqCSIST: Sequential Infrared Target Unmixing'
url: https://www.emergentmind.com/topics/seqcsist
type: topic
---

# SeqCSIST: Sequential Infrared Target Unmixing

Searching arXiv for the specified SeqCSIST paper to ground the response in the current preprint record.
Tool call: arxiv_search(query="2507.09556 SeqCSIST Sequential Closely-Spaced Infrared Small Target Unmixing", max_results=5)
SeqCSIST denotes an ecosystem centered on a new infrared sensing problem: **Sequential Closely-Spaced Infrared Small Target Unmixing**. In this setting, distant target groups are so densely packed that, because of the limitation of the optical lens focal length and the resolution of the infrared detector, they appear not as separately resolved objects but as a single mixed spot. SeqCSIST formalizes the task as detecting all targets in the form of **sub-pixel localization** from a highly dense CSIST group by leveraging multiple frames in a sequence, and couples that task with a sequential benchmark dataset, an evaluation toolkit, and a benchmark suite of 23 relevant methods. The accompanying method, **DeRefNet**, is a multi-frame, model-driven deep learning framework with a **Temporal Deformable Feature Alignment (TDFA)** module for adaptive inter-frame information aggregation [2507.09556].

## 1. Task definition and conceptual scope

A **Closely-Spaced Infrared Small Target (CSIST) group** is a cluster of multiple small infrared targets that are so close in angle or space that the optical system cannot resolve them individually. Because of finite focal length and aperture, diffraction through the Airy disk, and detector pixel size and sampling, the energy from several distant point targets is blurred and mixed into one fuzzy spot on the sensor. The paper states that, as inter-target distance decreases below one Rayleigh unit, multiple physical targets become one unresolved blob [2507.09556].

This formulation differs from conventional **Infrared Small Target Detection (IRSTD)**. Traditional IRSTD assumes a one-to-one mapping between a bright spot and a real target, and therefore typically outputs a box or segmentation mask. SeqCSIST instead asks two coupled questions inside a single mixed response: **how many targets are present** and **where they are located at sub-pixel precision**. The task is explicitly sequential rather than single-frame, so temporal subpixel motion becomes part of the signal rather than a nuisance variable.

The authors position sequential unmixing as necessary for early-warning and tracking systems in which aircraft, missiles, UAVs, or ships may appear as blended spots under optics and sensor limits. In that regime, detecting only the existence of a blob is insufficient; multiplicity and true positions are the operative variables. A plausible implication is that SeqCSIST shifts the problem class from pixel-scale detection toward a temporally conditioned inverse problem with sparse latent structure.

## 2. Imaging model and formal problem statement

The forward model begins with a high-resolution target distribution $H_{\text{GT}}$ and a low-resolution sensor measurement $L$. Each physical point target at sub-pixel coordinates $(x_t, y_t)$ with brightness $a_i$ is modeled by a Gaussian PSF,
$$
\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),
$$
and each sensor pixel integrates the PSF over its area. The blur-and-sample operator is collected into a sampling matrix $\Phi \in \mathbb{R}^{m \times n}$, yielding the imaging equation
$$
L = \Phi H_{\text{GT}}.
$$
The paper characterizes this as mathematically equivalent to a compressed sensing or inverse problem under a known operator $\Phi$ [2507.09556].

In the sequential setting, the input is a temporal window
$$
\{L_{t-N}, \ldots, L_t, \ldots, L_{t+N}\}, \quad L_t \in \mathbb{R}^{C \times H \times W},
$$
and the objective is to reconstruct, for the middle frame, a high-resolution unmixing map
$$
H_t \in \mathbb{R}^{C \times (cH) \times (cW)}, \quad c = 3,
$$
through
$$
H_t = f_{\text{DeRefNet}}(L_{t-N}, \ldots, L_{t+N}).
$$
Peaks in $H_t$ correspond to sub-pixel target locations. Multiple peaks inside the upsampled region associated with one low-resolution pixel indicate multiple closely-spaced targets.

The paper is explicit that this is not semantic segmentation and not ordinary single-frame IRSTD. It is a regression-style inverse problem over sequences, with evaluation carried out through detection-like metrics defined over target positions rather than box overlap. This suggests that temporal modeling is valuable precisely because the targets remain unresolved at the low-resolution frame level.

## 3. Dataset design, simulation principles, and annotation structure

SeqCSIST is a synthetic but physically grounded dataset designed specifically for sequential CSIST unmixing. Its headline statistics are fixed and central to the benchmark.

| Component | Specification |
|---|---|
| Trajectories | 5,000 |
| Frames per trajectory | 20 |
| Total frames | 100,000 |
| Low-resolution frame size | $11 \times 11$ |
| High-resolution ground truth | $33 \times 33$ |
| Standard target count | 2, 3, or 4 |
| Split | Train 70%, Val 15%, Test 15% |

For training and inference, the benchmark uses sliding sequences of **5 consecutive frames**. From 20 frames, it forms 16 overlapping subsequences: frames 1–5, 2–6, and so on through 16–20. The middle frame is the unmixing target [2507.09556].

The simulation is physically constrained. The paper states that approximately 84% of a point’s energy lies in the Airy disk, and approximates the Airy PSF with a 2D Gaussian whose variance is chosen so that the Airy disk radius is approximately $1.91\sigma$. In the dataset, diffusion variance is set to **0.5 pixel** in the low-resolution grid, target intensities are random in **[220, 250]**, and the blur and sampling are implemented through $\Phi$ computed with numerical integration using **dblquad**.

Target placement and motion are also specified. A reference point is defined in each frame, with each target keeping a fixed direction relative to that reference point throughout a trajectory. The reference point’s initial coordinate in frame 1 is $(x, f(x))$, where $x$ is an integer away from borders, and it moves along a quadratic, circular, or straight curve with a per-frame displacement of **$\pm 0.04$ pixel** along the x-axis. Each target begins at distance **0.3 pixel** from the reference point, with minimum pairwise distance approximately $0.3\sqrt{2}$ pixel, so all targets lie within the spatial extent of one low-resolution pixel while occupying different high-resolution pixel locations after unmixing. In each frame, each target moves away from the reference point by a small random amount in $(0, 0.0014]$ pixels.

For each frame, the dataset provides the low-resolution image $L$, the high-resolution ground truth $H_{\text{GT}}$, and XML files recording target coordinates in the low-resolution sub-pixel coordinate system together with intensities. The authors recommend treating SeqCSIST as a post-processing stage after a broader IRSTD detector: first crop an $11 \times 11$ window around a detected blob, then run the unmixing model on that crop.

## 4. Toolkit, benchmark protocol, and baseline families

The SeqCSIST ecosystem includes a toolkit that standardizes data access, visualization, evaluation, and baseline execution. The repository provides a dataset interface, display of low-resolution spots and corresponding high-resolution ground truth maps, an evaluation implementation for **CSO-mAP**, and baseline implementations for **23 methods** plus DeRefNet [2507.09556].

The benchmark metric is **CSO-mAP** (**Closely-Spaced Object mean Average Precision**), designed for sub-pixel unmixing. Each method produces a high-resolution unmixing map $H_t$ for the central frame; predicted targets are extracted as peaks with confidence scores and positions. A prediction is counted as a **True Positive** if it lies within a distance threshold $\delta_k$ of a ground-truth target and is the best match for that ground-truth point. Otherwise, it is a **False Positive**. At each threshold, the method computes a precision–recall curve and
$$
\text{AP}_{\delta_k} = \int_0^1 p(r)\,dr.
$$
The aggregate score is
$$
\text{CSO-mAP} = \frac{1}{K}\sum_{k=1}^K \text{AP}_{\delta_k}.
$$
The paper explicitly reports **CSO-mAP**, **AP\(_{05}\)**, **AP\(_{10}\)**, **AP\(_{15}\)**, **AP\(_{20}\)**, and **AP\(_{25}\)**, along with **FPS**, **number of parameters**, and **FLOPs**.

The 23 comparison methods are organized into three families. **Traditional optimization** includes **ISTA** and **BID**. **Image super-resolution networks** include **SRCNN**, **GMFN**, **DBPN**, **SRGAN**, **BSRGAN**, **ESRGAN**, **RDN**, **EDSR**, **ESPCN**, and **TDAN**. **Deep unfolding / learned inverse models** include **LIHT**, **LAMP**, **ISTA-Net**, **FISTA-Net**, **ISTA-Net+**, **ISTA-Net++**, **LISTA**, **USRNet**, **TiLISTA**, and **RPCANet**. This benchmark design makes the comparison space unusually broad: single-frame versus multi-frame, restoration-oriented versus inverse-problem-oriented, and generic architectures versus task-specific model-driven designs.

## 5. DeRefNet architecture and learning objective

DeRefNet is presented as a **multi-frame, model-driven deep unfolding network** specialized for sequential unmixing. It operates on 5 consecutive low-resolution frames, $\{L_{t-2}, L_{t-1}, L_t, L_{t+1}, L_{t+2}\}$, and consists of five stages: sampling and initialization, feature extraction via deep unfolding, temporal encoding, **Temporal Deformable Feature Alignment (TDFA)**, and aggregation with a tail network [2507.09556].

The initializer is linear and least-squares based:
$$
Q_{\text{init}} = \arg\min_Q \|QL - H\|^2 = L^\top (LL^\top)^{-1} H,
$$
followed by
$$
H_i^{(0)} = Q_{\text{init}} L_i.
$$
This is intended as a principled initialization analogous to the first step of ISTA rather than a generic learned upsampler.

Feature extraction is then unrolled through ISTA-like stages. For stage $k$,
$$
R_i^{(k)} = H_i^{(k-1)} - \rho \Phi^\top (\Phi H_i^{(k-1)} - L_i),
$$
and the refinement step uses a learned analysis transform $G^{(k)}$, soft-thresholding, and a learned approximate inverse $\tilde{G}^{(k)}$:
$$
H_i^{(k)} = \tilde{G}^{(k)}\big(\text{soft}( G^{(k)}(R_i^{(k)}), \theta^{(k)})\big).
$$
After unfolding, the network applies temporal positional encoding,
$$
T = \sigma(\text{MLP}(\text{Encoder}(t))), \qquad H' = H^{(k)} \cdot T,
$$
and then TDFA aligns reference-frame features to the middle frame. TDFA first performs **Selective Attention** to produce
$$
F_{\text{agg}i} = \text{SelectiveAttention}(F_i, F_t),
$$
then predicts deformable offsets
$$
\eta = \text{conv}(F_{\text{agg}i}),
$$
and samples dynamically from the reference feature map:
$$
H_i^{''} = \phi(F_i, \eta).
$$
At position $p$,
$$
H_i^{''}(p) = \sum_{k=1}^{K^2} \omega_k \cdot F_i(p + p_k + \Delta p_k),
$$
with bilinear interpolation used at fractional coordinates.

The final output is produced by concatenating aligned reference features with the middle-frame feature and passing them through a tail network consisting of **$n = 5$ residual blocks** and a final convolution. Training uses three losses: a **constraint loss** enforcing approximate invertibility of the learned transform pair, an **alignment loss** between aligned reference features and the middle-frame feature, and a **regression loss** between the predicted high-resolution map and ground truth. The total objective is
$$
\mathcal{L}_{\text{all}} = \beta \mathcal{L}_{\text{constraint}} + \gamma \mathcal{L}_{\text{align}} + \zeta \mathcal{L}_{\text{regression}}.
$$
The reported best hyperparameters are **$\beta = 0.01$**, **$\gamma = 0.01$**, and **$\zeta = 1$**, yielding **CSO-mAP = 51.55**.

## 6. Empirical results, interpretive significance, and nomenclature

On the SeqCSIST benchmark, DeRefNet is reported with **0.89M** parameters, **15.70G** FLOPs, **367 FPS**, **CSO-mAP 51.55**, **AP\(_{05}\) 1.00**, **AP\(_{10}\) 14.40**, **AP\(_{15}\) 54.90**, **AP\(_{20}\) 90.40**, and **AP\(_{25}\) 97.10**. The paper states that this is a **5.3% mAP improvement** over previous state-of-the-art approaches. Among baselines, **ISTA-Net+** is the strongest non-DeRefNet method at **51.02** CSO-mAP, while **TDAN** reaches **47.96**, **SRCNN** **49.64**, **GMFN** **50.94**, **DBPN** **50.40**, **EDSR** **50.19**, **ISTA-Net** **48.95**, **FISTA-Net** **50.61**, **USRNet** **49.25**, **RPCANet** **47.17**, **BID** **14.40**, and **ISTA** **10.72** [2507.09556].

The ablation studies identify several effects. Replacing TDAN’s residual-block extractor with the deep unfolding module raises CSO-mAP from **47.96** to **50.27**. With deep unfolding fixed, optical-flow alignment yields **50.55**, deformable alignment **50.67**, and deformable alignment plus time encoder **51.39**. A **Dynamic Deformable Alignment (DDA)** variant improves over the deformable-alignment baseline from **50.67 mAP, 6.44G FLOPs** to **51.09 mAP, 6.14G FLOPs**. Under a hybrid dataset in which SeqCSIST targets are composited onto real infrared backgrounds, performance changes from **51.55** to **47.48**. Under additive Gaussian noise, CSO-mAP changes from **51.55** at $\sigma = 0$ to **49.09** at $\sigma = 2$ and **47.23** at $\sigma = 5$. When target density is extended from **2–4** to **2–8** targets, performance changes from **51.55** to **49.79**.

Qualitative comparisons report that **ISTA** fails to separate targets and produces blurred blobs, **ISTA-Net** improves but still struggles as target count increases, and **DeRefNet** cleanly separates individual targets, recovering the number of sub-targets and their positions even when all sub-targets project into one low-resolution pixel. Comparisons with IRSTD methods such as **MSHNet**, **L2SKNet**, **IRSAM**, and **DNANet** show that blob-level detection does not resolve multiplicity or sub-pixel layout.

The stated limitations are equally important. The dataset is synthetic; there is no fully real annotated CSIST unmixing dataset. Extremely low SNR, strong clutter, atmospheric turbulence, target groups denser than 8 objects, and highly overlapping motion trajectories remain challenging. The forward operator $\Phi$ is assumed known and fixed, whereas real systems may exhibit spatially variant or unknown PSFs. The paper suggests future work on real or semi-real datasets, domain adaptation, self-supervision, physics-informed augmentation, alternative temporal models such as transformers, and scene-level integration with IRSTD pipelines.

A separate terminological note is warranted. In the wireless sensing thesis **"Statistical Analysis to Support CSI-Based Sensing Methods"** [2411.03203], the expression **“SeqCSIST-style methods”** is used informally for CSI treated as structured sequences evolving over time and frequency, with reference CSI, increment modeling, and distance-based characterization. That usage concerns sequential CSI-based sensing models rather than closely-spaced infrared small target unmixing. This suggests that, in contemporary arXiv usage, **SeqCSIST** is primarily the name of the infrared unmixing ecosystem, while a distinct and informal derivative phrase appears in an unrelated CSI-sensing context.

Source: https://www.emergentmind.com/topics/seqcsist