TSSTF: Temporally-Similar Structure-Aware ST Fusion

• The paper proposes TSSTF, an optimization-based framework that fuses spatiotemporal satellite images using temporally-guided TV and edge constraints to preserve fine structure under noise.
• It leverages a high-resolution reference alongside low-resolution observations to inject spatial details into temporally dense data while addressing Gaussian and sparse noise.
• Empirical results show TSSTF achieves superior PSNR and MSSIM, balancing noise suppression, edge sharpness, and realistic spectral changes.

Searching arXiv for the named TSSTF paper and closely related ST fusion work to ground the article and citations. Temporally-Similar Structure-Aware ST Fusion (TSSTF) is an optimization-based spatiotemporal fusion framework for satellite images that reconstructs a target high-spatial-resolution image from one high-resolution reference image, one low-resolution reference image, and one low-resolution target image while explicitly addressing realistic noise and preserving fine spatial structure (Isono et al., 15 Aug 2025). In this formulation, spatiotemporal fusion addresses the spatial–temporal trade-off between sensors such as Landsat and MODIS by injecting spatial detail from sparse high-resolution observations into temporally dense low-resolution observations. TSSTF is distinguished by two mechanisms—Temporally-Guided Total Variation (TGTV) and Temporally-Guided Edge Constraint (TGEC)—and by a constrained optimization program solved with a preconditioned primal-dual splitting algorithm (Isono et al., 15 Aug 2025).

1. Problem formulation and observational setting

TSSTF is defined for the setting in which a high-resolution image is available at a reference date, while only low-resolution observations are available at both the reference and target dates. Denoting the reference high-resolution image by $\mathbf{h}_r \in \mathbb{R}^{N_h B}$, the reference low-resolution image by $\mathbf{l}_r \in \mathbb{R}^{N_l B}$, and the target low-resolution image by $\mathbf{l}_t \in \mathbb{R}^{N_l B}$, the objective is to estimate the target high-resolution image $\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$ (Isono et al., 15 Aug 2025). The underlying super-resolution observation model is

$$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$

where $\mathbf{B}$ is a spatial blur, $\mathbf{S}$ is a downsampling operator, and $\mathbf{m}$ is a modeling error (Isono et al., 15 Aug 2025).

This setting is motivated by the complementarity of common Earth-observation sensors: Landsat offers high spatial resolution and low temporal resolution, while MODIS offers high temporal resolution and low spatial resolution (Isono et al., 15 Aug 2025). The method therefore targets applications that require both temporal density and spatial detail, including crop monitoring, land cover change, evapotranspiration, and ecosystem dynamics (Isono et al., 15 Aug 2025).

A defining assumption of TSSTF is temporal similarity of spatial structure. When reference and target dates are temporally close, their high-resolution images are assumed to share similar spatial structure, particularly edge locations such as field boundaries, roads, water bodies, and buildings, even when spectral intensities vary (Isono et al., 15 Aug 2025). This assumption is not merely heuristic: it is embedded directly into the regularization and constraint design.

2. Noise model and motivation for structure-aware fusion

TSSTF was proposed in response to a specific failure mode of prior noise-robust spatiotemporal fusion methods: robustness to noise often came at the expense of fine structure, causing oversmoothing, staircasing, and edge distortion (Isono et al., 15 Aug 2025). The framework explicitly models the fact that real satellite noise is not only Gaussian and pixelwise but also includes sparse corruption such as outliers, missing values, clouds, gaps, and corrupted lines. The noisy observations are written as

$$\mathbf{h} = \widehat{\mathbf{h}} + \mathbf{n}_h + \mathbf{s}_h,\qquad \mathbf{l} = \widehat{\mathbf{l}} + \mathbf{n}_l + \mathbf{s}_l,$$

where $\mathbf{n}_h,\mathbf{n}_l$ denote Gaussian noise and $\mathbf{l}_r \in \mathbb{R}^{N_l B}$0 denote sparse corruptions (Isono et al., 15 Aug 2025). The description further notes that high-resolution images usually have larger $\mathbf{l}_r \in \mathbb{R}^{N_l B}$1 and higher sparse noise ratio $\mathbf{l}_r \in \mathbb{R}^{N_l B}$2 than low-resolution images because they collect less light per pixel (Isono et al., 15 Aug 2025).

The immediate precursor ROSTF used standard total variation regularization, $\mathbf{l}_r \in \mathbb{R}^{N_l B}$3, to handle Gaussian and sparse noise, but standard TV treats all gradients uniformly and therefore tends to attenuate both noise gradients and true edges (Isono et al., 15 Aug 2025). By contrast, TSSTF uses temporal guidance from the reference high-resolution image to distinguish likely structural gradients from gradients that should be suppressed. In this sense, its “structure-aware” character is not a generic appeal to spatial context but a concrete reweighting of directional gradients according to a temporally nearby guide.

A common misconception is that TSSTF is simply another TV-regularized inverse problem. This is inaccurate. Classical TV promotes piecewise smoothness without discriminating edge directions, whereas TSSTF modifies both the regularizer and the inter-date constraint using guide-derived directional weights (Isono et al., 15 Aug 2025). The practical consequence is that smooth regions are regularized strongly, while gradients aligned with likely structure are penalized weakly or not at all.

3. Core mechanisms: TGTV and TGEC

TSSTF introduces two coupled mechanisms: Temporally-Guided Total Variation and Temporally-Guided Edge Constraint (Isono et al., 15 Aug 2025). Both are derived from a guide image built from the noisy high-resolution reference.

The guide image is a denoised single-band proxy for spatial structure:

$\mathbf{l}_r \in \mathbb{R}^{N_l B}$4

where $\mathbf{l}_r \in \mathbb{R}^{N_l B}$5 is band $\mathbf{l}_r \in \mathbb{R}^{N_l B}$6 of the reference high-resolution image and $\mathbf{l}_r \in \mathbb{R}^{N_l B}$7 is a spatial median filter (Isono et al., 15 Aug 2025). From this guide, finite-difference operators $\mathbf{l}_r \in \mathbb{R}^{N_l B}$8 in four directions define directional weights

$\mathbf{l}_r \in \mathbb{R}^{N_l B}$9

with $\mathbf{l}_t \in \mathbb{R}^{N_l B}$0 controlling sensitivity (Isono et al., 15 Aug 2025). If the guide is smooth in direction $\mathbf{l}_t \in \mathbb{R}^{N_l B}$1, the weight is close to $\mathbf{l}_t \in \mathbb{R}^{N_l B}$2; if the guide contains an edge, the weight becomes small. TSSTF further sharpens this selectivity by setting to zero the $\mathbf{l}_t \in \mathbb{R}^{N_l B}$3 smallest directional weights at each pixel, so only the strongest directions remain active (Isono et al., 15 Aug 2025).

For an image $\mathbf{l}_t \in \mathbb{R}^{N_l B}$4, TGTV is defined as

$\mathbf{l}_t \in \mathbb{R}^{N_l B}$5

Unlike isotropic TV, this regularizer is directionally modulated by the temporally guided weight matrix $\mathbf{l}_t \in \mathbb{R}^{N_l B}$6 (Isono et al., 15 Aug 2025). In guide-smooth regions, gradients are strongly penalized; in guide-edge directions, the penalty is weak, so boundaries and small structures are preserved.

TGEC is the companion constraint that couples the denoised reference and estimated target high-resolution images. A naive edge-consistency term such as

$\mathbf{l}_t \in \mathbb{R}^{N_l B}$7

would force gradient magnitudes to be similar everywhere and would therefore suppress legitimate temporal spectral changes (Isono et al., 15 Aug 2025). TSSTF instead imposes

$\mathbf{l}_t \in \mathbb{R}^{N_l B}$8

This weighted form strongly constrains smooth regions to remain smooth and aligned, while relaxing the constraint in edge regions where gradient magnitudes may change even when edge locations persist (Isono et al., 15 Aug 2025). The paper reports that the mixed $\mathbf{l}_t \in \mathbb{R}^{N_l B}$9 norm performs best for TGEC in terms of PSNR and stability and recommends $\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$0 (Isono et al., 15 Aug 2025).

Together, TGTV and TGEC implement the central TSSTF principle: temporal similarity is used not to force pixelwise identity across dates, but to preserve geometric structure while allowing radiometric variation.

4. Constrained optimization and solution algorithm

TSSTF jointly estimates the denoised high-resolution reference image $\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$1, the target high-resolution image $\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$2, and sparse noise components $\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$3 by solving the constrained optimization problem

$\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$4

(Isono et al., 15 Aug 2025)

Only the TGTV terms appear in the objective; the remaining model components are encoded as constraints (Isono et al., 15 Aug 2025). This separation is significant because it decouples parameter roles and simplifies tuning. The bandwise mean constraint enforces spectral brightness consistency between target low-resolution and target high-resolution images, while the $\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$5 and $\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$6 constraints encode Gaussian fidelity and sparse corruption bounds, respectively (Isono et al., 15 Aug 2025).

The optimization is reformulated with auxiliary variables into a generic convex primal-dual form and solved by a preconditioned primal-dual splitting (P-PDS) method (Isono et al., 15 Aug 2025). The primal and dual updates are standard proximal steps, with the required proximity operators including mixed $\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$7 shrinkage and projections onto hyperslabs, $\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$8 balls, $\widehat{\mathbf{h}_t} \in \mathbb{R}^{N_h B}$9 balls, and $$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$0 balls (Isono et al., 15 Aug 2025). Step sizes are chosen automatically by operator-norm-based variable preconditioning (OVDP):

$$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$1

(Isono et al., 15 Aug 2025)

A further feature is the adaptive TGEC threshold

$$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$2

which scales with current reference-edge strength and the average magnitude of temporal change observed in low-resolution data (Isono et al., 15 Aug 2025). The paper reports that $$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$3 stabilizes after approximately $$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$4 iterations, after which the algorithm behaves like P-PDS on a fixed constraint (Isono et al., 15 Aug 2025). Typical runs use about $$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$5–$$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$6 iterations with stopping criterion $$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$7 (Isono et al., 15 Aug 2025).

5. Empirical behavior, benchmarks, and recommended parameterization

The evaluation uses five sites (Site1–Site5) with real Landsat high-resolution imagery and resampled MODIS low-resolution imagery (Isono et al., 15 Aug 2025). High-resolution resolution is $$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$8, while the HR:LR scale ratio is $$\widehat{\mathbf{l}} = \mathbf{S}\mathbf{B}\widehat{\mathbf{h}} + \mathbf{m},$$9 for Site1 and Site2 and $\mathbf{B}$0 for Sites3–5 (Isono et al., 15 Aug 2025). Simulated low-resolution data are also generated by the forward model $\mathbf{B}$1 to isolate fusion performance without cross-sensor inconsistencies (Isono et al., 15 Aug 2025).

Four high-resolution noise cases are considered while low-resolution images are kept noise-free: no noise; Gaussian noise only with $\mathbf{B}$2; Gaussian plus sparse noise with $\mathbf{B}$3; and Gaussian plus stronger sparse noise with $\mathbf{B}$4 (Isono et al., 15 Aug 2025). The method is compared with STARFM, VIPSTF, RobOt, SwinSTFM, RSFN, and ROSTF (Isono et al., 15 Aug 2025). Performance is assessed by PSNR and MSSIM (Isono et al., 15 Aug 2025).

The reported findings are consistent across simulated and real data. Under noise-free conditions, TSSTF performs comparably to state-of-the-art methods and often attains top MSSIM with near-top PSNR (Isono et al., 15 Aug 2025). Under noisy conditions, it consistently achieves the highest PSNR across all sites and noise cases, and highest or second-highest MSSIM (Isono et al., 15 Aug 2025). Qualitative comparisons attribute distinct failure modes to competing methods: VIPSTF shows strong blurring, RSFN and SwinSTFM exhibit inaccurate spectral changes, STARFM and RobOt propagate noise, and ROSTF suppresses noise but loses structure and introduces spectral artifacts near edges (Isono et al., 15 Aug 2025). TSSTF is described as producing the best balance of noise suppression, edge sharpness, and realistic spectral change (Isono et al., 15 Aug 2025).

The paper also provides a recommended parameter set intended to work consistently across sites and noise conditions:

Parameter	Definition or tested setting	Recommendation
$\mathbf{B}$5	Weight sensitivity in $\mathbf{B}$6; tested from $\mathbf{B}$7 to $\mathbf{B}$8	$\mathbf{B}$9
$\mathbf{S}$0	Number of smallest directional weights set to zero	$\mathbf{S}$1
$\mathbf{S}$2	Norm in TGEC; compared $\mathbf{S}$3, $\mathbf{S}$4, $\mathbf{S}$5	$\mathbf{S}$6
$\mathbf{S}$7	Coefficient in adaptive $\mathbf{S}$8	$\mathbf{S}$9
$\mathbf{m}$0	TGTV balance between reference and target	$\mathbf{m}$1

(Isono et al., 15 Aug 2025)

Additional parameters are derived from observed quantities and assumed noise statistics rather than tuned freely: $\mathbf{m}$2, $\mathbf{m}$3, $\mathbf{m}$4, $\mathbf{m}$5, and $\mathbf{m}$6 (Isono et al., 15 Aug 2025). This parameterization is one of the framework’s practical contributions because it reduces the degree of manual tuning ordinarily associated with constrained variational fusion.

6. Nomenclature, related uses, and cross-domain interpretations

In the literature represented here, TSSTF has one explicit and canonical meaning: the satellite-image fusion framework described above (Isono et al., 15 Aug 2025). A recurrent source of confusion is that closely related phrases—especially “temporally similar,” “structure-aware,” and “spatio-temporal fusion”—also appear in other domains, but often as descriptive syntheses rather than as formally named modules.

For road-network trajectories, ST2Vec separates trajectories into spatial and temporal components, models road structure with Node2Vec and GCN, models time with Time2Vec-like embeddings, LSTM, and self-attention, and fuses both streams with a spatio-temporal co-attention fusion module (Fang et al., 2021). For visual state space models, Spatial-Mamba introduces a structure-aware state fusion equation that restores neighborhood connectivity directly in latent state space through multi-scale dilated depth-wise convolutions (Xiao et al., 2024). For remote-sensing change detection, STNet combines bi-temporal features through a temporal feature fusion module and recovers fine spatial details through cross-scale attention in a spatial feature fusion module (Ma et al., 2023). For action recognition, STFN studies feature-level fusion of appearance and motion streams across entire videos using residual inception blocks to capture local and global temporal dynamics (Cho et al., 2019). In smart mobility, an overview paper explicitly states that it does not introduce a module named TSSTF, but it synthesizes mechanisms—meta-learning, graph attention, autocorrelation attention, and multimodal fusion—that correspond closely to temporally-similar, structure-aware spatiotemporal fusion (Wang, 2024).

This broader record suggests a common design principle, though this is an inference rather than a paper-defined equivalence: temporal similarity becomes most effective when it guides structural modeling instead of being appended after it. In the satellite-image TSSTF formulation, that principle appears in its most explicit convex form, because the guide-derived operator $\mathbf{m}$7 shapes both regularization and cross-time coupling (Isono et al., 15 Aug 2025). By contrast, the related systems above instantiate analogous ideas through co-attention, graph neural encoding, gated temporal fusion, or state-space neighborhood fusion rather than through temporally guided variational constraints (Fang et al., 2021, Xiao et al., 2024, Ma et al., 2023, Cho et al., 2019, Wang, 2024).

Another important distinction concerns limitations and scope. Satellite-image TSSTF assumes a single reference date, temporally similar edge locations, matching spectral bands between sensors, and a Gaussian-plus-sparse noise model (Isono et al., 15 Aug 2025). The paper identifies several plausible extensions, including multi-reference TSSTF, sensor-agnostic or multimodal fusion, deep learning integration via plug-and-play priors or loss terms, nonlinear temporal models, and spatially adaptive noise models (Isono et al., 15 Aug 2025). A plausible implication is that these directions would move TSSTF toward the broader family of structure-aware spatiotemporal fusion architectures already explored in trajectories, remote sensing change detection, video modeling, and smart mobility, while retaining the explicit interpretability of TGTV and TGEC.