---
title: 'TGPNet: Task-Guided Remote Sensing Restoration'
url: https://www.emergentmind.com/topics/tgpnet
type: topic
---

# TGPNet: Task-Guided Remote Sensing Restoration

Searching arXiv for TGPNet and closely related name-confusion papers to ground the article.
{"query":"TGPNet arXiv Task-Guided Prompting Network remote sensing image restoration", "max_results": 10}
{"query":"TrafficGPT TGPNet arXiv", "max_results": 10}
{"query":"Trainable Projected Gradient Method TPGM arXiv 2303.10720", "max_results": 5}
TGPNet, short for **Task-Guided Prompting Network**, is a unified remote sensing image restoration framework designed to handle heterogeneous degradations and sensing modalities within a single shared architecture. Its target setting is unified RSIR, in which a degraded input image \(I_{\text{in}}\) is restored to \(I_{\text{out}}\) while the model is conditioned on an explicit task prompt. The framework covers **denoising**, **cloud removal**, **shadow removal**, **deblurring**, and **SAR despeckling**, spanning **RGB optical**, **multispectral optical**, **SAR**, and **thermal infrared** data. The core contribution is **Task-Guided Prompting (TGP)**, which converts learnable task-specific prompts into hierarchical decoder-side feature modulations inside a Restormer-based encoder-decoder, enabling one model to behave as multiple task-adaptive restorers while keeping a shared parameterization [2604.02742].

## 1. Problem setting and design objective

TGPNet is formulated as a unified mapping
\[
I_{\text{out}} = \mathcal{M}(I_{\text{in}}, F_{\text{TP}}; \Theta),
\]
where \(F_{\text{TP}}\) is the task prompt and \(\Theta\) denotes the shared model parameters. The problem is not merely multi-task restoration within a single image domain. The paper explicitly frames it as **cross-domain, cross-modality, multi-degradation restoration**, because the degradations differ not only visually but physically: optical denoising is additive, cloud and shadow removal involve occlusion and illumination effects, thermal deblurring alters spatial structure, and SAR despeckling addresses coherent-imaging artifacts.

The five supported tasks are fixed and explicit: optical denoising with additive Gaussian noise, cloud removal on RGB and multispectral imagery, shadow removal on optical imagery, deblurring on thermal infrared imagery, and SAR despeckling. A central operational premise is that real remote sensing pipelines rarely process a single homogeneous degradation type in isolation. TGPNet therefore replaces task-specific expert models with a single architecture conditioned by task-aware prompts.

This design also imposes a clear inference assumption: TGPNet is **non-blind**. It requires an explicit task prior at test time, rather than inferring the degradation type autonomously. The paper presents this as both a practical mechanism for unification and a limitation for real deployments [2604.02742].

## 2. Task-Guided Prompting mechanism

The defining mechanism is **Task-Guided Prompting (TGP)**, which consists of two modules: **Learnable Task-Specific Embedding (LTSE)** and **Hierarchical Feature Modulation (HFM)**. LTSE takes a learnable task prompt \(F_{TP}\) and transforms it into a compact embedding \(F_E\):
\[
F_{E} = \text{GAP}\left(\sigma\left(W_3 \left(\sigma\left(W_2 \left(\sigma\left(W_1 \left(F_{TP}\right)\right)\right)\right)\right)\right)\right),
\]
where \(W_1\) is a \(7 \times 7\) convolution, \(W_2\) and \(W_3\) are \(3 \times 3\) convolutions, all with stride \(2\), \(\sigma\) is ReLU, and GAP is global average pooling. The paper characterizes this embedding as a latent degradation-aware descriptor rather than a textual or semantic-language prompt.

HFM maps \(F_E\) into channel-wise affine modulation parameters:
\[
\gamma = \text{Linear}_1(F_E), \qquad \beta = \text{Linear}_2(F_E),
\]
and applies them to decoder features \(F_{Di}\) via
\[
F_{Mi} = \gamma \circledstar F_{Di} + \beta,
\]
where \(\circledstar\) denotes channel-wise multiplication. This is a FiLM-style modulation mechanism. Its importance is structural: the backbone remains shared, while task prompts alter decoder feature statistics so that reconstruction behavior becomes degradation-specific.

The modulation is hierarchical rather than single-shot. TGP modules are inserted at **six points**: the **four decoder stages**, **one after the refinement module**, and **one after the final terminal convolution**. If the base width is \(C\), then decoder stage \(i\) uses channel dimension
\[
C_{Di} = 2^{(i-1)}C.
\]
This placement reflects the paper’s explicit design choice that the encoder can remain generic while the decoder becomes the locus of task specialization [2604.02742].

## 3. Backbone architecture and training protocol

TGPNet adopts a **transformer-based U-shaped encoder-decoder** adapted from **Restormer**. The input degraded image \(I_{in} \in \mathbb{R}^{H \times W \times C_{in}}\) first passes through a \(3 \times 3\) convolution to produce shallow features \(F_S \in \mathbb{R}^{H \times W \times C}\), with \(C=48\) in the reported experiments. The encoder has four stages:
- \(E_1\): \(H \times W \times C\)
- \(E_2\): \(H/2 \times W/2 \times 2C\)
- \(E_3\): \(H/4 \times W/4 \times 4C\)
- \(E_4\): \(H/8 \times W/8 \times 8C\)

Each stage uses transformer blocks and downsampling by convolution plus pixel unshuffle. The decoder mirrors this structure with upsampling by convolution plus pixel shuffle, skip concatenation, channel reduction by \(1 \times 1\) convolution, refinement blocks, and TGP modulation after each decoder stage. The stage-wise block layout is explicitly reported as:
- \(E_1\): 1 Transformer Block, \(N_h=2\)
- \(E_2\): 2 Transformer Blocks, \(N_h=4\)
- \(E_3\): 2 Transformer Blocks, \(N_h=8\)
- \(E_4\): 4 Transformer Blocks, \(N_h=8\)
- \(D_4\): 4 Transformer Blocks, then TGP
- \(D_3\): 2 Transformer Blocks, then TGP
- \(D_2\): 2 Transformer Blocks, then TGP
- \(D_1\): 1 Transformer Block, then TGP
- Refinement: 2 Transformer Blocks, then TGP
- Residual generation: terminal convolution, then TGP

The output is residual:
\[
I_{out} = I_{in} + I_r.
\]
This residual form later becomes important in the discussion of SAR limitations.

Training is fully joint across tasks. All five degradation datasets are merged into a single training pool; each mini-batch is sampled from that pool and paired with the corresponding task prompt. The sole optimization objective is an \(\mathcal{L}_1\) reconstruction loss:
\[
\mathcal{L}_1 = \frac{1}{HWC}\sum_{h=1}^{H}\sum_{w=1}^{W}\sum_{c=1}^{C}\left|I_{\text{out}}^{(h,w,c)} - I_{\text{gt}}^{(h,w,c)}\right|.
\]
No auxiliary task-specific, perceptual, adversarial, or spectral losses are introduced. Optimization uses **AdamW** with \(\beta_1=0.9\), \(\beta_2=0.999\), **450 epochs**, **batch size 14**, **2 NVIDIA RTX 4090 GPUs**, initial learning rate \(2\times 10^{-4}\), cosine annealing with restarts, and EMA with decay \(\alpha=0.999\). Training patches are random crops of \(128 \times 128\) with random flips and random rotations [2604.02742].

## 4. Unified benchmark and empirical performance

The paper introduces the **URSIR** benchmark for unified remote sensing image restoration. It includes:
- **UCMLUD** for denoising
- **RICE1**, **RICE2**, and **SEN12MS-CR** for declouding
- **SRD** and **UAV-TSS** for deshadowing
- **NRD** for SAR despeckling
- **HIT-UAV** for thermal deblurring

The evaluation proceeds progressively from **2-task optical**, to **3-task optical**, to **4-task dual-domain**, and finally **5-task tri-domain** restoration. The principal metrics are **PSNR** and **SSIM**, while single-task declouding also reports **MAE** and **SAM**.

On the unified settings, TGPNet reports the best average performance in each progression. In the **2-task** setting, it reaches **38.06 dB / 0.9354**, compared with **CR-former** at **36.61 dB / 0.9320**. In the **3-task** setting, it reaches **32.90 dB / 0.9051**, compared with **CR-former** at **32.36 dB / 0.9019**. In the **4-task** setting, it achieves **31.52 dB / 0.8874**, ahead of **CR-former** at **31.20 / 0.8826**, **AdaIR** at **30.73 / 0.8830**, **MOCE-IR** at **30.26 / 0.8838**, and **PromptIR** at **30.42 / 0.8818**. In the full **5-task** setting, TGPNet again leads with **31.14 dB / 0.8890**, compared with **CR-former** at **31.03 / 0.8870**, **AdaIR** at **29.85 / 0.8799**, **PromptIR** at **29.50 / 0.8742**, and **MOCE-IR** at **28.92 / 0.8694** [2604.02742].

Task-level numbers reinforce the claim that unification does not collapse single-task competence. In the 5-task setting, the reported values include **RICE1 35.59 / 0.9591**, **RICE2 35.93 / 0.9130**, **SEN12MS-CR 29.61 / 0.8963**, **SRD 27.97 / 0.8739**, **UCMLUD average 31.55 / 0.8723**, **NRD 24.95 / 0.8294**, and **HIT-UAV average 32.35 / 0.8789**. On single-task RICE2 declouding, TGPNet attains **PSNR 35.89**, **SSIM 0.9132**, **MAE 0.0168**, and **SAM 1.27**, exceeding or matching specialized declouding baselines such as **ACA-CRNet**, **Restormer**, and **CR-former**.

A crucial nuance is SAR performance. The paper explicitly notes that TGPNet does **not** always achieve the best SAR PSNR among all compared methods, even though it remains strong overall and often competitive in structural quality. This becomes part of the method’s limitation profile rather than a contradiction of its unified restoration claim.

## 5. Ablations, feature analysis, and unseen composite degradations

The clearest architectural ablation compares the shared backbone with and without TGP. On the 4-task setting, the baseline backbone reaches **31.38 / 0.8865**, adding a ReLU-based variant yields **31.50 / 0.8867**, and the proposed TGP reaches **31.52 / 0.8874**. The gains are modest in aggregate but consistent across tasks, including **RICE1 35.29 \(\to\) 35.73**, **RICE2 36.07 \(\to\) 36.35**, **SRD 28.19 \(\to\) 28.40**, and **SAR SSIM 0.8284 \(\to\) 0.8329**. The paper also reports that adding ReLU after affine modulation brings negligible benefit, supporting the claim that affine conditioning is sufficient because nonlinearity already exists in LTSE and the backbone [2604.02742].

A more distinctive analysis concerns representation structure. Using K-means on decoder stage-2 features, TGP increases **Silhouette** from **0.1605** to **0.4388**, **CH index** from **163.11** to **572.76**, **Dunn** from **0.1450** to **0.2754**, **ARI** from **0.4837** to **0.9610**, **AMI** from **0.6348** to **0.9279**, and **FMI** from **0.6407** to **0.9736**. In the 5-task setting, comparing decoder features before and after TGP gives **ARI 0.707 \(\to\) 0.9124** and **FMI 0.7783 \(\to\) 0.9338**. This indicates that TGPNet does not merely improve restoration metrics; it induces strongly task-separable decoder representations.

The prompt strategy ablation is subtler. Random learnable prompts slightly underperform CLIP-based prompts on some RGB tasks but do better on **SEN12MS-CR** multispectral declouding (**29.61** vs **29.50** PSNR) and **SAR despeckling** (**24.95** vs **24.93** PSNR). The overall averages are **31.14 / 0.8890** for random learnable prompts and **31.25 / 0.8898** for CLIP-based prompts. The paper interprets this not as a strict dominance result but as a modality trade-off: language priors help RGB-centric tasks, whereas RS-specific learnable prompts are better aligned with non-RGB domains.

Generalization to unseen **composite degradations** is one of the most important empirical findings. TGPNet is trained only on single degradations, then tested on unseen mixtures such as Denoising+Declouding and Denoising+Deblurring+Deshadowing. Two inference strategies are evaluated. The **direct** strategy averages task embeddings; the **sequential** strategy applies the model multiple times with different task prompts in a chosen order. The direct strategy fails badly: on Denoising+Declouding for RICE1 at \(\sigma=15\), **TGPNet Direct** yields **18.81 / 0.6741**, whereas **TGPNet Sequential** reaches **30.67 / 0.8489**. Averaged over all two-composite settings, **Ours (Direct)** gives **19.04 / 0.5736**, while **Ours (Sequential)** gives **29.76 / 0.8138**. Over three-composite settings, **Ours (Direct)** gives **17.20 / 0.2656**, while **Ours (Sequential)** gives **24.25 / 0.7242**. This suggests that TGPNet’s strongest compositional generalization is not one-shot mixed-degradation restoration, but the sequential composition of known task-conditioned restoration skills.

## 6. Efficiency, limitations, and nomenclature

For \(256 \times 256\) inputs on RTX 3090, TGPNet reports **21.27M** parameters, **71.42G** FLOPs, **1643 MB** GPU memory, and **60.84 ms** inference time. Relative to unified baselines, it is not the smallest model, but it is among the best performance-efficiency trade-offs and is reported as faster than several high-performing competitors such as **PromptIR**, **AdaIR**, **PromptHSI**, and **CR-former**. Its efficiency profile is therefore practical rather than minimalistic [2604.02742].

The limitations are explicit. First, TGPNet is **non-blind** and requires a task prompt at inference. Second, joint optimization across additive optical noise, occlusion-type degradations, TIR blur, and SAR speckle can induce **negative transfer**. Third, the residual output
\[
I_{out} = I_{in} + I_r
\]
is naturally aligned with additive degradations, which the paper identifies as a reason for weaker SAR PSNR under multiplicative speckle corruption. Fourth, direct handling of unseen composite degradations is poor; the successful strategy is sequential prompting rather than prompt averaging.

A separate source of confusion is nomenclature. Several arXiv works have nearby acronyms or similar titles but are not TGPNet: **“TrafficGPT”** [2403.05822], **“Trainable Projected Gradient Method”** [2303.10720], **“TGP: Two-modal occupancy prediction with 3D Gaussian and sparse points for 3D Environment Awareness”** [2503.09941], and **“Tensor Gaussian Processes”** [2510.13772] all describe distinct methods and do not use **TGPNet** as their formal model name. In contrast, the remote sensing restoration model discussed here explicitly uses **TGPNet** to denote a unified RSIR framework built around **Task-Guided Prompting** [2604.02742].

Source: https://www.emergentmind.com/topics/tgpnet