Temporally-Guided Edge Constraint (TGEC)

• TGEC is a structure-aware constraint that preserves fine spatial details by enforcing edge consistency across temporally adjacent high-resolution images in satellite fusion.
• It employs a spatially adaptive weighting operator derived from a denoised reference image to balance edge matching and spectral variations, avoiding oversmoothing.
• The method utilizes a mixed ℓ1,2 norm with adaptive thresholding to robustly manage noise while maintaining structural fidelity in spatiotemporal image fusion.

Temporally-Guided Edge Constraint (TGEC) is a structure-aware constraint introduced in the spatiotemporal fusion framework "Temporally-Similar Structure-Aware ST fusion" (TSSTF). Its purpose is to enforce consistency in edge locations between two temporally adjacent high-resolution images while allowing for spectral variations, thereby transferring spatial structure across time without imposing radiometric equality. In the broader literature, the closest direct analogue is found in "Flow-edge Guided Video Completion," where completed motion edges constrain optical-flow completion and subsequently guide temporal propagation, although that work does not use the term TGEC explicitly (Isono et al., 15 Aug 2025, Gao et al., 2020).

1. Problem setting and structural premise

TGEC is defined in a one-reference-date satellite-image fusion setting. The available observations are a high-resolution image on a past reference date, denoted $\mathbf{h}_r \in \mathbb{R}^{N_h B}$, a low-resolution image on the same reference date, denoted $\mathbf{l}_r \in \mathbb{R}^{N_l B}$, and a low-resolution image on the target date, denoted $\mathbf{l}_t \in \mathbb{R}^{N_l B}$. The unknowns are the noiseless high-resolution reference image $\widetilde{\mathbf{h}}_r$, the noiseless high-resolution target image $\widetilde{\mathbf{h}}_t$, and sparse-noise variables for the observed images (Isono et al., 15 Aug 2025).

The key prior is temporal adjacency. If the reference date and target date are close, the corresponding high-resolution images are expected to share similar spatial structure even when their spectral brightness changes over time. TGEC is designed precisely for that regime: it encodes structural similarity across dates while avoiding the stronger and generally incorrect assumption that corresponding pixel intensities should match.

This modeling choice is particularly relevant to noisy spatiotemporal fusion. The TSSTF formulation is motivated by the observation that existing noise-robust fusion methods often fail to capture fine spatial structure, leading to oversmoothing and artifacts. TGEC addresses that limitation by constraining how local spatial differences in the two high-resolution estimates may disagree. The intended effect is not generic denoising by itself, but preservation of fine structure during fusion.

The underlying assumption can fail when genuine structural change occurs between dates. The source material explicitly identifies land-cover conversion, flooding or drying that moves boundaries, harvest, construction, fire scars, cloud or shadow contamination not removed, and geometric misregistration between dates or sensors as situations in which edge-location consistency becomes less appropriate. This suggests that TGEC is most defensible when temporal separation is short and dominant scene geometry is stable.

2. Mathematical definition

TGEC is built on four directional difference operators. For an image $\mathbf{x}\in\mathbb{R}^{WHB}$, the operators $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$ are defined by

$$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$

$$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$

These are stacked as

$$\mathbf{D} := (\mathbf{D}_1^{\top}\ \mathbf{D}_2^{\top}\ \mathbf{D}_3^{\top}\ \mathbf{D}_4^{\top})^{\top} \in \mathbb{R}^{4N_h B \times N_h B}.$$

A conventional inter-date edge constraint would take the form

$\mathbf{l}_r \in \mathbb{R}^{N_l B}$0

where $\mathbf{l}_r \in \mathbb{R}^{N_l B}$1 is a norm and $\mathbf{l}_r \in \mathbb{R}^{N_l B}$2 is a tolerance. TGEC modifies this by inserting a spatially adaptive diagonal weighting operator $\mathbf{l}_r \in \mathbb{R}^{N_l B}$3, derived from a temporally adjacent high-resolution reference image: $\mathbf{l}_r \in \mathbb{R}^{N_l B}$4

The paper evaluates $\mathbf{l}_r \in \mathbb{R}^{N_l B}$5, and reports that the mixed $\mathbf{l}_r \in \mathbb{R}^{N_l B}$6-norm gives the best performance. The mixed norm is defined as

$\mathbf{l}_r \in \mathbb{R}^{N_l B}$7

When applied to $\mathbf{l}_r \in \mathbb{R}^{N_l B}$8, the discrepancy is measured in a vector-valued manner rather than independently for each band (Isono et al., 15 Aug 2025).

The distinction between TGEC and the naive constraint is central. The unweighted form tries to make edge responses themselves similar. TGEC instead makes that similarity spatially selective. In smooth regions, the weighting is large, so agreement of local differences is enforced strongly. Around edges and other non-smooth regions, the weighting becomes small, so the model tolerates discrepancy in edge intensity. This is why the paper characterizes TGEC as enforcing consistency of edge locations while allowing for spectral variations.

3. Construction of the temporal guidance

The temporal guidance enters through the weight matrix $\mathbf{l}_r \in \mathbb{R}^{N_l B}$9. Because the observed high-resolution reference image $\mathbf{l}_t \in \mathbb{R}^{N_l B}$0 may be noisy, the guide image is first constructed as a denoised grayscale image

$\mathbf{l}_t \in \mathbb{R}^{N_l B}$1

where $\mathbf{l}_t \in \mathbb{R}^{N_l B}$2 denotes the $\mathbf{l}_t \in \mathbb{R}^{N_l B}$3-th spectral band and $\mathbf{l}_t \in \mathbb{R}^{N_l B}$4 is a median filter applied bandwise. Temporal guidance therefore comes from the observed reference high-resolution image after median filtering each band and averaging across bands.

Directional weights are then computed for each pixel location $\mathbf{l}_t \in \mathbb{R}^{N_l B}$5 and direction $\mathbf{l}_t \in \mathbb{R}^{N_l B}$6: $\mathbf{l}_t \in \mathbb{R}^{N_l B}$7 Large guide-image differences make $\mathbf{l}_t \in \mathbb{R}^{N_l B}$8 small, and small guide-image differences make $\mathbf{l}_t \in \mathbb{R}^{N_l B}$9 large. To sharpen structural selectivity, the method introduces a parameter $\widetilde{\mathbf{h}}_r$0: at each $\widetilde{\mathbf{h}}_r$1, the $\widetilde{\mathbf{h}}_r$2 smallest weights among $\widetilde{\mathbf{h}}_r$3 are set to zero. Experimentally, the paper recommends $\widetilde{\mathbf{h}}_r$4.

The full weighting operator is assembled directionwise. For each direction,

$\widetilde{\mathbf{h}}_r$5

which is then replicated across spectral bands,

$\widetilde{\mathbf{h}}_r$6

and finally stacked into

$\widetilde{\mathbf{h}}_r$7

This construction is continuous rather than binary. The paper explicitly notes that TGEC does not use a separate binary edge detector, Canny or Sobel-style thresholding, or an explicit structural-similarity term inside TGEC itself. The weighting acts instead as an anisotropic, guide-dependent differential mask. In TSSTF, the same $\widetilde{\mathbf{h}}_r$8 is shared by TGEC and Temporally-Guided Total Variation (TGTV), so the reference-date guide simultaneously controls where smoothing should be encouraged and where cross-date edge consistency should be relaxed (Isono et al., 15 Aug 2025).

4. TGEC within the TSSTF optimization framework

TGEC is not introduced as a penalty term in the objective. It appears as a hard constraint inside the TSSTF optimization problem: $\widetilde{\mathbf{h}}_r$9 subject to

$\widetilde{\mathbf{h}}_t$0

together with average-preservation, observation-fidelity, and sparse-noise constraints for $\widetilde{\mathbf{h}}_t$1, $\widetilde{\mathbf{h}}_t$2, and $\widetilde{\mathbf{h}}_t$3. In this formulation, the first two objective terms are TGTV penalties and the first constraint is TGEC. The paper uses $\widetilde{\mathbf{h}}_t$4 (Isono et al., 15 Aug 2025).

The hard-constraint interpretation is maintained in the optimization reformulation. Using the indicator function

$\widetilde{\mathbf{h}}_t$5

TGEC becomes the indicator of the norm ball

$\widetilde{\mathbf{h}}_t$6

applied to the weighted inter-date differential discrepancy: $\widetilde{\mathbf{h}}_t$7 This is not a soft relaxation. Feasibility with respect to the norm ball remains mandatory.

A notable implementation detail is that the algorithm does not keep $\widetilde{\mathbf{h}}_t$8 fixed. At iteration $\widetilde{\mathbf{h}}_t$9, it sets

$\mathbf{x}\in\mathbb{R}^{WHB}$0

The authors explain that the appropriate tolerance depends both on temporal spectral change, estimated from the low-resolution images, and on scene edge strength and density, estimated from the progressively denoised reference high-resolution estimate. The recommended value is $\mathbf{x}\in\mathbb{R}^{WHB}$1.

Optimization is performed by a preconditioned primal-dual splitting method with operator-norm-based variable-wise diagonal preconditioning. With auxiliary variable

$\mathbf{x}\in\mathbb{R}^{WHB}$2

TGEC appears as

$\mathbf{x}\in\mathbb{R}^{WHB}$3

In the primal updates, the coupling enters with opposite signs through $\mathbf{x}\in\mathbb{R}^{WHB}$4: $\mathbf{x}\in\mathbb{R}^{WHB}$5

$\mathbf{x}\in\mathbb{R}^{WHB}$6

The TGEC-related dual forward step is

$\mathbf{x}\in\mathbb{R}^{WHB}$7

followed by projection onto the $\mathbf{x}\in\mathbb{R}^{WHB}$8-norm ball via the conjugate proximal. The paper provides the projection machinery for $\mathbf{x}\in\mathbb{R}^{WHB}$9, $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$0, and the mixed $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$1-ball, the last being especially relevant because $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$2 is the recommended norm.

5. Empirical behavior, recommended settings, and limitations

The paper does not provide an ablation that turns TGEC on and off independently while keeping TGTV fixed, so it does not report an isolated TGEC-only numerical effect. However, several empirical results are directly tied to TGEC. The most specific is a norm-sensitivity study. Average PSNR across the four simulated noise cases is reported as follows: Case1, $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$3; Case2, $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$4; Case3, $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$5; Case4, $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$6. Across sites, the averages are Site1, $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$7; Site2, $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$8; Site3, $$\mathbf{D}_1,\mathbf{D}_2,\mathbf{D}_3,\mathbf{D}_4 \in \mathbb{R}^{WHB\times WHB}$$9; Site4, $$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$0; Site5, $$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$1. On that basis, the recommended TGEC norm is the mixed $$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$2-norm (Isono et al., 15 Aug 2025).

A second TGEC-specific study concerns the adaptive threshold coefficient $$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$3. The paper tests $$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$4 and reports consistently strong performance around $$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$5 across sites and noise levels. It also states that TSSTF performs comparably to state-of-the-art methods under noise-free conditions and outperforms them under noisy conditions. Since the closest robust optimization baseline is ROSTF, which uses a non-weighted edge constraint, these gains support the value of the temporally guided weighted formulation.

The reported recommended settings relevant to TGEC are summarized below.

Parameter	Recommended value
$$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$6	$$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$7
$$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$8	$$[\mathbf{D}_1 \mathbf{x}]_{i,j,b} = x_{i+1,j,b} - x_{i,j,b}, \qquad [\mathbf{D}_2 \mathbf{x}]_{i,j,b} = x_{i+1,j-1,b} - x_{i,j,b},$$9
$$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$0	$$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$1
$$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$2	$$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$3
$$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$4	$$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$5

The convergence discussion is also relevant. The authors note that if $$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$6 were fixed at an intermediate value, the problem would be convex and the algorithm would be guaranteed to converge to its optimum for that fixed constraint. Empirically, $$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$7 stabilizes after about 1000 iterations, the update error $$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$8 decreases almost monotonically, and PSNR plateaus after about 2000 iterations. This is the main convergence evidence tied specifically to the adaptive TGEC implementation.

The scope of TGEC is bounded by its modeling assumptions. The weighting softens the constraint near non-smooth regions, but the prior still presumes that temporal adjacency implies structural similarity. Where genuine structural change dominates, the constraint may become less appropriate. This suggests that TGEC is best understood as a temporally local structural prior rather than a universal inter-date invariance principle.

6. Relation to edge-guided temporal methods in video completion

Although TGEC is explicitly formulated in TSSTF for satellite-image fusion, a closely related mechanism appears in flow-based video completion. "Flow-edge Guided Video Completion" first extracts and completes motion edges and then uses them to guide piecewise-smooth flow completion with sharp edges. In that method, motion edges are edges in the optical-flow map rather than image-intensity edges, and the completed edge map determines where smoothness is enforced and where it is not. The completed flow is obtained by minimizing finite-difference gradients away from predicted edges, subject to equality with observed flow outside the mask, producing piecewise-smooth flow with sharp discontinuities (Gao et al., 2020).

That work also contains an explicitly temporal component. It propagates content through adjacent-frame flow links and supplements them with non-local temporal neighbors to three distant frames: the first, middle, and last frames. The motivation is that motion boundaries can form “impenetrable barriers” for purely local temporal propagation. Non-local links create shortcuts across those barriers, and validity is controlled by forward-backward flow consistency with threshold $$[\mathbf{D}_3 \mathbf{x}]_{i,j,b} = x_{i,j-1,b} - x_{i,j,b}, \qquad [\mathbf{D}_4 \mathbf{x}]_{i,j,b} = x_{i-1,j-1,b} - x_{i,j,b}.$$9 pixels. Candidate fusion uses

$$\mathbf{D} := (\mathbf{D}_1^{\top}\ \mathbf{D}_2^{\top}\ \mathbf{D}_3^{\top}\ \mathbf{D}_4^{\top})^{\top} \in \mathbb{R}^{4N_h B \times N_h B}.$$0

and the final frame is reconstructed in the gradient domain by a Poisson solve.

The relation to TGEC is functional rather than terminological. The paper itself does not name a “Temporally-Guided Edge Constraint,” and it does not provide a single joint optimization over edge completion, flow completion, and temporal propagation. Instead, it implements a modular pipeline in which completed motion edges act as structural constraints on flow completion, and the resulting edge-preserving flow field guides temporal propagation both locally and non-locally. This suggests a broader interpretation of TGEC-like methodology: edge-conditioned differential constraints can serve as the structural substrate on which temporal guidance operates, even when the formal constraint is expressed in a domain other than satellite-image fusion.