SeqCSIST: Sequential Infrared Target Unmixing

• SeqCSIST is a framework for sequentially unmixing closely-spaced infrared small targets, enabling sub-pixel localization from multi-frame inputs.
• It leverages a deep unfolding model with Temporal Deformable Feature Alignment to improve detection performance by 5.3% mAP over previous methods.
• The ecosystem includes a synthetic, physics-based dataset, an evaluation toolkit, and a benchmark suite of 23 baseline methods for rigorous performance analysis.

Searching arXiv for the specified SeqCSIST paper to ground the response in the current preprint record. Tool call: arxiv_search(query="(Zhai et al., 13 Jul 2025) SeqCSIST Sequential Closely-Spaced Infrared Small Target Unmixing", max_results=5) SeqCSIST denotes an ecosystem centered on a new infrared sensing problem: Sequential Closely-Spaced Infrared Small Target Unmixing. In this setting, distant target groups are so densely packed that, because of the limitation of the optical lens focal length and the resolution of the infrared detector, they appear not as separately resolved objects but as a single mixed spot. SeqCSIST formalizes the task as detecting all targets in the form of sub-pixel localization from a highly dense CSIST group by leveraging multiple frames in a sequence, and couples that task with a sequential benchmark dataset, an evaluation toolkit, and a benchmark suite of 23 relevant methods. The accompanying method, DeRefNet, is a multi-frame, model-driven deep learning framework with a Temporal Deformable Feature Alignment (TDFA) module for adaptive inter-frame information aggregation (Zhai et al., 13 Jul 2025).

1. Task definition and conceptual scope

A Closely-Spaced Infrared Small Target (CSIST) group is a cluster of multiple small infrared targets that are so close in angle or space that the optical system cannot resolve them individually. Because of finite focal length and aperture, diffraction through the Airy disk, and detector pixel size and sampling, the energy from several distant point targets is blurred and mixed into one fuzzy spot on the sensor. The paper states that, as inter-target distance decreases below one Rayleigh unit, multiple physical targets become one unresolved blob (Zhai et al., 13 Jul 2025).

This formulation differs from conventional Infrared Small Target Detection (IRSTD). Traditional IRSTD assumes a one-to-one mapping between a bright spot and a real target, and therefore typically outputs a box or segmentation mask. SeqCSIST instead asks two coupled questions inside a single mixed response: how many targets are present and where they are located at sub-pixel precision. The task is explicitly sequential rather than single-frame, so temporal subpixel motion becomes part of the signal rather than a nuisance variable.

The authors position sequential unmixing as necessary for early-warning and tracking systems in which aircraft, missiles, UAVs, or ships may appear as blended spots under optics and sensor limits. In that regime, detecting only the existence of a blob is insufficient; multiplicity and true positions are the operative variables. A plausible implication is that SeqCSIST shifts the problem class from pixel-scale detection toward a temporally conditioned inverse problem with sparse latent structure.

2. Imaging model and formal problem statement

The forward model begins with a high-resolution target distribution $H_{\text{GT}}$ and a low-resolution sensor measurement $L$. Each physical point target at sub-pixel coordinates $(x_t, y_t)$ with brightness $a_i$ is modeled by a Gaussian PSF,

$$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$

and each sensor pixel integrates the PSF over its area. The blur-and-sample operator is collected into a sampling matrix $\Phi \in \mathbb{R}^{m \times n}$, yielding the imaging equation

$L = \Phi H_{\text{GT}}.$

The paper characterizes this as mathematically equivalent to a compressed sensing or inverse problem under a known operator $\Phi$ (Zhai et al., 13 Jul 2025).

In the sequential setting, the input is a temporal window

$$\{L_{t-N}, \ldots, L_t, \ldots, L_{t+N}\}, \quad L_t \in \mathbb{R}^{C \times H \times W},$$

and the objective is to reconstruct, for the middle frame, a high-resolution unmixing map

$$H_t \in \mathbb{R}^{C \times (cH) \times (cW)}, \quad c = 3,$$

through

$L$0

Peaks in $L$1 correspond to sub-pixel target locations. Multiple peaks inside the upsampled region associated with one low-resolution pixel indicate multiple closely-spaced targets.

The paper is explicit that this is not semantic segmentation and not ordinary single-frame IRSTD. It is a regression-style inverse problem over sequences, with evaluation carried out through detection-like metrics defined over target positions rather than box overlap. This suggests that temporal modeling is valuable precisely because the targets remain unresolved at the low-resolution frame level.

3. Dataset design, simulation principles, and annotation structure

SeqCSIST is a synthetic but physically grounded dataset designed specifically for sequential CSIST unmixing. Its headline statistics are fixed and central to the benchmark.

Component	Specification
Trajectories	5,000
Frames per trajectory	20
Total frames	100,000
Low-resolution frame size	$L$2
High-resolution ground truth	$L$3
Standard target count	2, 3, or 4
Split	Train 70%, Val 15%, Test 15%

For training and inference, the benchmark uses sliding sequences of 5 consecutive frames. From 20 frames, it forms 16 overlapping subsequences: frames 1–5, 2–6, and so on through 16–20. The middle frame is the unmixing target (Zhai et al., 13 Jul 2025).

The simulation is physically constrained. The paper states that approximately 84% of a point’s energy lies in the Airy disk, and approximates the Airy PSF with a 2D Gaussian whose variance is chosen so that the Airy disk radius is approximately $L$4. In the dataset, diffusion variance is set to 0.5 pixel in the low-resolution grid, target intensities are random in [220, 250], and the blur and sampling are implemented through $L$5 computed with numerical integration using dblquad.

Target placement and motion are also specified. A reference point is defined in each frame, with each target keeping a fixed direction relative to that reference point throughout a trajectory. The reference point’s initial coordinate in frame 1 is $L$6, where $L$7 is an integer away from borders, and it moves along a quadratic, circular, or straight curve with a per-frame displacement of $L$8 pixel along the x-axis. Each target begins at distance 0.3 pixel from the reference point, with minimum pairwise distance approximately $L$9 pixel, so all targets lie within the spatial extent of one low-resolution pixel while occupying different high-resolution pixel locations after unmixing. In each frame, each target moves away from the reference point by a small random amount in $(x_t, y_t)$0 pixels.

For each frame, the dataset provides the low-resolution image $(x_t, y_t)$1, the high-resolution ground truth $(x_t, y_t)$2, and XML files recording target coordinates in the low-resolution sub-pixel coordinate system together with intensities. The authors recommend treating SeqCSIST as a post-processing stage after a broader IRSTD detector: first crop an $(x_t, y_t)$3 window around a detected blob, then run the unmixing model on that crop.

4. Toolkit, benchmark protocol, and baseline families

The SeqCSIST ecosystem includes a toolkit that standardizes data access, visualization, evaluation, and baseline execution. The repository provides a dataset interface, display of low-resolution spots and corresponding high-resolution ground truth maps, an evaluation implementation for CSO-mAP, and baseline implementations for 23 methods plus DeRefNet (Zhai et al., 13 Jul 2025).

The benchmark metric is CSO-mAP (Closely-Spaced Object mean Average Precision), designed for sub-pixel unmixing. Each method produces a high-resolution unmixing map $(x_t, y_t)$4 for the central frame; predicted targets are extracted as peaks with confidence scores and positions. A prediction is counted as a True Positive if it lies within a distance threshold $(x_t, y_t)$5 of a ground-truth target and is the best match for that ground-truth point. Otherwise, it is a False Positive. At each threshold, the method computes a precision–recall curve and

$(x_t, y_t)$6

The aggregate score is

$(x_t, y_t)$7

The paper explicitly reports CSO-mAP, AP$(x_t, y_t)$8, AP$(x_t, y_t)$9, AP$a_i$0, AP$a_i$1, and AP$a_i$2, along with FPS, number of parameters, and FLOPs.

The 23 comparison methods are organized into three families. Traditional optimization includes ISTA and BID. Image super-resolution networks include SRCNN, GMFN, DBPN, SRGAN, BSRGAN, ESRGAN, RDN, EDSR, ESPCN, and TDAN. Deep unfolding / learned inverse models include LIHT, LAMP, ISTA-Net, FISTA-Net, ISTA-Net+, ISTA-Net++, LISTA, USRNet, TiLISTA, and RPCANet. This benchmark design makes the comparison space unusually broad: single-frame versus multi-frame, restoration-oriented versus inverse-problem-oriented, and generic architectures versus task-specific model-driven designs.

5. DeRefNet architecture and learning objective

DeRefNet is presented as a multi-frame, model-driven deep unfolding network specialized for sequential unmixing. It operates on 5 consecutive low-resolution frames, $a_i$3, and consists of five stages: sampling and initialization, feature extraction via deep unfolding, temporal encoding, Temporal Deformable Feature Alignment (TDFA), and aggregation with a tail network (Zhai et al., 13 Jul 2025).

The initializer is linear and least-squares based:

$a_i$4

followed by

$a_i$5

This is intended as a principled initialization analogous to the first step of ISTA rather than a generic learned upsampler.

Feature extraction is then unrolled through ISTA-like stages. For stage $a_i$6,

$a_i$7

and the refinement step uses a learned analysis transform $a_i$8, soft-thresholding, and a learned approximate inverse $a_i$9:

$$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$0

After unfolding, the network applies temporal positional encoding,

$$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$1

and then TDFA aligns reference-frame features to the middle frame. TDFA first performs Selective Attention to produce

$$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$2

then predicts deformable offsets

$$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$3

and samples dynamically from the reference feature map:

$$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$4

At position $$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$5,

$$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$6

with bilinear interpolation used at fractional coordinates.

The final output is produced by concatenating aligned reference features with the middle-frame feature and passing them through a tail network consisting of $$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$7 residual blocks and a final convolution. Training uses three losses: a constraint loss enforcing approximate invertibility of the learned transform pair, an alignment loss between aligned reference features and the middle-frame feature, and a regression loss between the predicted high-resolution map and ground truth. The total objective is

$$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$8

The reported best hyperparameters are $$\text{PSF}(x,y; x_t, y_t) = a_i \cdot \frac{1}{2\pi\sigma^2} \exp\!\left(-\frac{(x-x_t)^2 + (y-y_t)^2}{2\sigma^2}\right),$$9, $\Phi \in \mathbb{R}^{m \times n}$0, and $\Phi \in \mathbb{R}^{m \times n}$1, yielding CSO-mAP = 51.55.

6. Empirical results, interpretive significance, and nomenclature

On the SeqCSIST benchmark, DeRefNet is reported with 0.89M parameters, 15.70G FLOPs, 367 FPS, CSO-mAP 51.55, AP$\Phi \in \mathbb{R}^{m \times n}$2 1.00, AP$\Phi \in \mathbb{R}^{m \times n}$3 14.40, AP$\Phi \in \mathbb{R}^{m \times n}$4 54.90, AP$\Phi \in \mathbb{R}^{m \times n}$5 90.40, and AP$\Phi \in \mathbb{R}^{m \times n}$6 97.10. The paper states that this is a 5.3% mAP improvement over previous state-of-the-art approaches. Among baselines, ISTA-Net+ is the strongest non-DeRefNet method at 51.02 CSO-mAP, while TDAN reaches 47.96, SRCNN 49.64, GMFN 50.94, DBPN 50.40, EDSR 50.19, ISTA-Net 48.95, FISTA-Net 50.61, USRNet 49.25, RPCANet 47.17, BID 14.40, and ISTA 10.72 (Zhai et al., 13 Jul 2025).

The ablation studies identify several effects. Replacing TDAN’s residual-block extractor with the deep unfolding module raises CSO-mAP from 47.96 to 50.27. With deep unfolding fixed, optical-flow alignment yields 50.55, deformable alignment 50.67, and deformable alignment plus time encoder 51.39. A Dynamic Deformable Alignment (DDA) variant improves over the deformable-alignment baseline from 50.67 mAP, 6.44G FLOPs to 51.09 mAP, 6.14G FLOPs. Under a hybrid dataset in which SeqCSIST targets are composited onto real infrared backgrounds, performance changes from 51.55 to 47.48. Under additive Gaussian noise, CSO-mAP changes from 51.55 at $\Phi \in \mathbb{R}^{m \times n}$7 to 49.09 at $\Phi \in \mathbb{R}^{m \times n}$8 and 47.23 at $\Phi \in \mathbb{R}^{m \times n}$9. When target density is extended from 2–4 to 2–8 targets, performance changes from 51.55 to 49.79.

Qualitative comparisons report that ISTA fails to separate targets and produces blurred blobs, ISTA-Net improves but still struggles as target count increases, and DeRefNet cleanly separates individual targets, recovering the number of sub-targets and their positions even when all sub-targets project into one low-resolution pixel. Comparisons with IRSTD methods such as MSHNet, L2SKNet, IRSAM, and DNANet show that blob-level detection does not resolve multiplicity or sub-pixel layout.

The stated limitations are equally important. The dataset is synthetic; there is no fully real annotated CSIST unmixing dataset. Extremely low SNR, strong clutter, atmospheric turbulence, target groups denser than 8 objects, and highly overlapping motion trajectories remain challenging. The forward operator $L = \Phi H_{\text{GT}}.$0 is assumed known and fixed, whereas real systems may exhibit spatially variant or unknown PSFs. The paper suggests future work on real or semi-real datasets, domain adaptation, self-supervision, physics-informed augmentation, alternative temporal models such as transformers, and scene-level integration with IRSTD pipelines.

A separate terminological note is warranted. In the wireless sensing thesis "Statistical Analysis to Support CSI-Based Sensing Methods" (Tonini, 2024), the expression “SeqCSIST-style methods” is used informally for CSI treated as structured sequences evolving over time and frequency, with reference CSI, increment modeling, and distance-based characterization. That usage concerns sequential CSI-based sensing models rather than closely-spaced infrared small target unmixing. This suggests that, in contemporary arXiv usage, SeqCSIST is primarily the name of the infrared unmixing ecosystem, while a distinct and informal derivative phrase appears in an unrelated CSI-sensing context.