TGPNet: Task-Guided Remote Sensing Restoration

• TGPNet is a unified restoration framework that uses task-guided prompting to modulate a transformer-based encoder-decoder for multi-degradation remote sensing imagery.
• It employs a learnable task-specific embedding and hierarchical feature modulation to adapt features for tasks like optical denoising, cloud removal, shadow removal, thermal deblurring, and SAR despeckling.
• TGPNet supports both single and composite degradation restoration, achieving leading PSNR and SSIM benchmarks on diverse unified RSIR datasets.

Searching arXiv for TGPNet and closely related name-confusion papers to ground the article. {"query":"TGPNet arXiv Task-Guided Prompting Network remote sensing image restoration", "max_results": 10} {"query":"TrafficGPT TGPNet arXiv", "max_results": 10} {"query":"Trainable Projected Gradient Method TPGM arXiv (Tian et al., 2023)", "max_results": 5} TGPNet, short for Task-Guided Prompting Network, is a unified remote sensing image restoration framework designed to handle heterogeneous degradations and sensing modalities within a single shared architecture. Its target setting is unified RSIR, in which a degraded input image $I_{\text{in}}$ is restored to $I_{\text{out}}$ while the model is conditioned on an explicit task prompt. The framework covers denoising, cloud removal, shadow removal, deblurring, and SAR despeckling, spanning RGB optical, multispectral optical, SAR, and thermal infrared data. The core contribution is Task-Guided Prompting (TGP), which converts learnable task-specific prompts into hierarchical decoder-side feature modulations inside a Restormer-based encoder-decoder, enabling one model to behave as multiple task-adaptive restorers while keeping a shared parameterization (Huang et al., 3 Apr 2026).

1. Problem setting and design objective

TGPNet is formulated as a unified mapping

$$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$

where $F_{\text{TP}}$ is the task prompt and $\Theta$ denotes the shared model parameters. The problem is not merely multi-task restoration within a single image domain. The paper explicitly frames it as cross-domain, cross-modality, multi-degradation restoration, because the degradations differ not only visually but physically: optical denoising is additive, cloud and shadow removal involve occlusion and illumination effects, thermal deblurring alters spatial structure, and SAR despeckling addresses coherent-imaging artifacts.

The five supported tasks are fixed and explicit: optical denoising with additive Gaussian noise, cloud removal on RGB and multispectral imagery, shadow removal on optical imagery, deblurring on thermal infrared imagery, and SAR despeckling. A central operational premise is that real remote sensing pipelines rarely process a single homogeneous degradation type in isolation. TGPNet therefore replaces task-specific expert models with a single architecture conditioned by task-aware prompts.

This design also imposes a clear inference assumption: TGPNet is non-blind. It requires an explicit task prior at test time, rather than inferring the degradation type autonomously. The paper presents this as both a practical mechanism for unification and a limitation for real deployments (Huang et al., 3 Apr 2026).

2. Task-Guided Prompting mechanism

The defining mechanism is Task-Guided Prompting (TGP), which consists of two modules: Learnable Task-Specific Embedding (LTSE) and Hierarchical Feature Modulation (HFM). LTSE takes a learnable task prompt $F_{TP}$ and transforms it into a compact embedding $F_E$: $$F_{E} = \text{GAP}\left(\sigma\left(W_3 \left(\sigma\left(W_2 \left(\sigma\left(W_1 \left(F_{TP}\right)\right)\right)\right)\right)\right)\right),$$ where $W_1$ is a $7 \times 7$ convolution, $I_{\text{out}}$0 and $I_{\text{out}}$1 are $I_{\text{out}}$2 convolutions, all with stride $I_{\text{out}}$3, $I_{\text{out}}$4 is ReLU, and GAP is global average pooling. The paper characterizes this embedding as a latent degradation-aware descriptor rather than a textual or semantic-language prompt.

HFM maps $I_{\text{out}}$5 into channel-wise affine modulation parameters: $I_{\text{out}}$6 and applies them to decoder features $I_{\text{out}}$7 via

$I_{\text{out}}$8

where $I_{\text{out}}$9 denotes channel-wise multiplication. This is a FiLM-style modulation mechanism. Its importance is structural: the backbone remains shared, while task prompts alter decoder feature statistics so that reconstruction behavior becomes degradation-specific.

The modulation is hierarchical rather than single-shot. TGP modules are inserted at six points: the four decoder stages, one after the refinement module, and one after the final terminal convolution. If the base width is $$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$0, then decoder stage $$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$1 uses channel dimension

$$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$2

This placement reflects the paper’s explicit design choice that the encoder can remain generic while the decoder becomes the locus of task specialization (Huang et al., 3 Apr 2026).

3. Backbone architecture and training protocol

TGPNet adopts a transformer-based U-shaped encoder-decoder adapted from Restormer. The input degraded image $$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$3 first passes through a $$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$4 convolution to produce shallow features $$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$5, with $$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$6 in the reported experiments. The encoder has four stages:

• $$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$7: $$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$8
• $$I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),$$9: $F_{\text{TP}}$0
• $F_{\text{TP}}$1: $F_{\text{TP}}$2
• $F_{\text{TP}}$3: $F_{\text{TP}}$4

Each stage uses transformer blocks and downsampling by convolution plus pixel unshuffle. The decoder mirrors this structure with upsampling by convolution plus pixel shuffle, skip concatenation, channel reduction by $F_{\text{TP}}$5 convolution, refinement blocks, and TGP modulation after each decoder stage. The stage-wise block layout is explicitly reported as:

• $F_{\text{TP}}$6: 1 Transformer Block, $F_{\text{TP}}$7
• $F_{\text{TP}}$8: 2 Transformer Blocks, $F_{\text{TP}}$9
• $\Theta$0: 2 Transformer Blocks, $\Theta$1
• $\Theta$2: 4 Transformer Blocks, $\Theta$3
• $\Theta$4: 4 Transformer Blocks, then TGP
• $\Theta$5: 2 Transformer Blocks, then TGP
• $\Theta$6: 2 Transformer Blocks, then TGP
• $\Theta$7: 1 Transformer Block, then TGP
• Refinement: 2 Transformer Blocks, then TGP
• Residual generation: terminal convolution, then TGP

The output is residual: $\Theta$8 This residual form later becomes important in the discussion of SAR limitations.

Training is fully joint across tasks. All five degradation datasets are merged into a single training pool; each mini-batch is sampled from that pool and paired with the corresponding task prompt. The sole optimization objective is an $\Theta$9 reconstruction loss: $F_{TP}$0 No auxiliary task-specific, perceptual, adversarial, or spectral losses are introduced. Optimization uses AdamW with $F_{TP}$1, $F_{TP}$2, 450 epochs, batch size 14, 2 NVIDIA RTX 4090 GPUs, initial learning rate $F_{TP}$3, cosine annealing with restarts, and EMA with decay $F_{TP}$4. Training patches are random crops of $F_{TP}$5 with random flips and random rotations (Huang et al., 3 Apr 2026).

4. Unified benchmark and empirical performance

The paper introduces the URSIR benchmark for unified remote sensing image restoration. It includes:

• UCMLUD for denoising
• RICE1, RICE2, and SEN12MS-CR for declouding
• SRD and UAV-TSS for deshadowing
• NRD for SAR despeckling
• HIT-UAV for thermal deblurring

The evaluation proceeds progressively from 2-task optical, to 3-task optical, to 4-task dual-domain, and finally 5-task tri-domain restoration. The principal metrics are PSNR and SSIM, while single-task declouding also reports MAE and SAM.

On the unified settings, TGPNet reports the best average performance in each progression. In the 2-task setting, it reaches 38.06 dB / 0.9354, compared with CR-former at 36.61 dB / 0.9320. In the 3-task setting, it reaches 32.90 dB / 0.9051, compared with CR-former at 32.36 dB / 0.9019. In the 4-task setting, it achieves 31.52 dB / 0.8874, ahead of CR-former at 31.20 / 0.8826, AdaIR at 30.73 / 0.8830, MOCE-IR at 30.26 / 0.8838, and PromptIR at 30.42 / 0.8818. In the full 5-task setting, TGPNet again leads with 31.14 dB / 0.8890, compared with CR-former at 31.03 / 0.8870, AdaIR at 29.85 / 0.8799, PromptIR at 29.50 / 0.8742, and MOCE-IR at 28.92 / 0.8694 (Huang et al., 3 Apr 2026).

Task-level numbers reinforce the claim that unification does not collapse single-task competence. In the 5-task setting, the reported values include RICE1 35.59 / 0.9591, RICE2 35.93 / 0.9130, SEN12MS-CR 29.61 / 0.8963, SRD 27.97 / 0.8739, UCMLUD average 31.55 / 0.8723, NRD 24.95 / 0.8294, and HIT-UAV average 32.35 / 0.8789. On single-task RICE2 declouding, TGPNet attains PSNR 35.89, SSIM 0.9132, MAE 0.0168, and SAM 1.27, exceeding or matching specialized declouding baselines such as ACA-CRNet, Restormer, and CR-former.

A crucial nuance is SAR performance. The paper explicitly notes that TGPNet does not always achieve the best SAR PSNR among all compared methods, even though it remains strong overall and often competitive in structural quality. This becomes part of the method’s limitation profile rather than a contradiction of its unified restoration claim.

5. Ablations, feature analysis, and unseen composite degradations

The clearest architectural ablation compares the shared backbone with and without TGP. On the 4-task setting, the baseline backbone reaches 31.38 / 0.8865, adding a ReLU-based variant yields 31.50 / 0.8867, and the proposed TGP reaches 31.52 / 0.8874. The gains are modest in aggregate but consistent across tasks, including RICE1 35.29 $F_{TP}$6 35.73, RICE2 36.07 $F_{TP}$7 36.35, SRD 28.19 $F_{TP}$8 28.40, and SAR SSIM 0.8284 $F_{TP}$9 0.8329. The paper also reports that adding ReLU after affine modulation brings negligible benefit, supporting the claim that affine conditioning is sufficient because nonlinearity already exists in LTSE and the backbone (Huang et al., 3 Apr 2026).

A more distinctive analysis concerns representation structure. Using K-means on decoder stage-2 features, TGP increases Silhouette from 0.1605 to 0.4388, CH index from 163.11 to 572.76, Dunn from 0.1450 to 0.2754, ARI from 0.4837 to 0.9610, AMI from 0.6348 to 0.9279, and FMI from 0.6407 to 0.9736. In the 5-task setting, comparing decoder features before and after TGP gives ARI 0.707 $F_E$0 0.9124 and FMI 0.7783 $F_E$1 0.9338. This indicates that TGPNet does not merely improve restoration metrics; it induces strongly task-separable decoder representations.

The prompt strategy ablation is subtler. Random learnable prompts slightly underperform CLIP-based prompts on some RGB tasks but do better on SEN12MS-CR multispectral declouding (29.61 vs 29.50 PSNR) and SAR despeckling (24.95 vs 24.93 PSNR). The overall averages are 31.14 / 0.8890 for random learnable prompts and 31.25 / 0.8898 for CLIP-based prompts. The paper interprets this not as a strict dominance result but as a modality trade-off: language priors help RGB-centric tasks, whereas RS-specific learnable prompts are better aligned with non-RGB domains.

Generalization to unseen composite degradations is one of the most important empirical findings. TGPNet is trained only on single degradations, then tested on unseen mixtures such as Denoising+Declouding and Denoising+Deblurring+Deshadowing. Two inference strategies are evaluated. The direct strategy averages task embeddings; the sequential strategy applies the model multiple times with different task prompts in a chosen order. The direct strategy fails badly: on Denoising+Declouding for RICE1 at $F_E$2, TGPNet Direct yields 18.81 / 0.6741, whereas TGPNet Sequential reaches 30.67 / 0.8489. Averaged over all two-composite settings, Ours (Direct) gives 19.04 / 0.5736, while Ours (Sequential) gives 29.76 / 0.8138. Over three-composite settings, Ours (Direct) gives 17.20 / 0.2656, while Ours (Sequential) gives 24.25 / 0.7242. This suggests that TGPNet’s strongest compositional generalization is not one-shot mixed-degradation restoration, but the sequential composition of known task-conditioned restoration skills.

6. Efficiency, limitations, and nomenclature

For $F_E$3 inputs on RTX 3090, TGPNet reports 21.27M parameters, 71.42G FLOPs, 1643 MB GPU memory, and 60.84 ms inference time. Relative to unified baselines, it is not the smallest model, but it is among the best performance-efficiency trade-offs and is reported as faster than several high-performing competitors such as PromptIR, AdaIR, PromptHSI, and CR-former. Its efficiency profile is therefore practical rather than minimalistic (Huang et al., 3 Apr 2026).

The limitations are explicit. First, TGPNet is non-blind and requires a task prompt at inference. Second, joint optimization across additive optical noise, occlusion-type degradations, TIR blur, and SAR speckle can induce negative transfer. Third, the residual output

$F_E$4

is naturally aligned with additive degradations, which the paper identifies as a reason for weaker SAR PSNR under multiplicative speckle corruption. Fourth, direct handling of unseen composite degradations is poor; the successful strategy is sequential prompting rather than prompt averaging.

A separate source of confusion is nomenclature. Several arXiv works have nearby acronyms or similar titles but are not TGPNet: “TrafficGPT” (Qu et al., 2024), “Trainable Projected Gradient Method” (Tian et al., 2023), “TGP: Two-modal occupancy prediction with 3D Gaussian and sparse points for 3D Environment Awareness” (Chen et al., 13 Mar 2025), and “Tensor Gaussian Processes” (Yuan et al., 15 Oct 2025) all describe distinct methods and do not use TGPNet as their formal model name. In contrast, the remote sensing restoration model discussed here explicitly uses TGPNet to denote a unified RSIR framework built around Task-Guided Prompting (Huang et al., 3 Apr 2026).