---
title: 'Semi-UFormer: Uncertainty-Aware Dehazing'
url: https://www.emergentmind.com/topics/semi-uformer
type: topic
---

# Semi-UFormer: Uncertainty-Aware Dehazing

Semi-UFormer is a semi-supervised, uncertainty-aware transformer network for image dehazing. Its central objective is to bridge the domain gap between synthetic and real hazy images, leveraging knowledge distillation and uncertainty modeling to enhance restoration fidelity. The framework explicitly models pixel-level aleatoric uncertainty and uses it to guide dehazing, achieving leading quantitative and qualitative results on both synthetic benchmarks and real-world images [2210.16057].

## 1. Architectural Overview

Semi-UFormer employs a two-stage teacher–student paradigm based on a U-Net-style backbone augmented with Mix DehazeFormer Blocks (MDB). Both teacher and student networks consist of five hierarchical stages, each incorporating the following components:

- **Shallow Feature Extraction:** A $3 \times 3$ convolution $C_3(\cdot)$ for preliminary feature extraction.
- **Mix DehazeFormer Block (MDB):** Composed of $n$ DehazeFormer transformer layers $DF(\cdot)$ followed by a residual convolutional block $RB(\cdot)$. This design enables effective global information fusion via
  $$
  F_\text{global} = RB\left(DF(F_\text{shallow})_n\right) + DF(F_\text{shallow})_n
  $$
- **Reconstruction Head:** Another $3 \times 3$ convolution with a pixel-shuffle operation $P(\cdot)$, which generates the final dehazed output, $J = P(C_3(F_\text{global}))$.

The teacher network is first trained to produce a coarse dehazed image and a corresponding uncertainty map. The student, initialized from the teacher's trained weights, freezes the uncertainty estimation branch and is retrained with both labeled (synthetic) and unlabeled (real) data.

## 2. Knowledge Distillation and Semi-Supervised Learning

The core learning objective decouples into supervised and unsupervised components for both teacher and student networks:

- **Teacher Supervised Loss:** Combines uncertainty-estimation loss $L_{ue}$ and an adversarial reconstruction loss $L_a$:
  $$
  L_{ts} = \lambda_1 L_{ue} + \lambda_2 L_a
  $$
- **Teacher Unsupervised Loss:** Incorporates identity loss $L_{ide}$, dark-channel prior loss $L_{dc}$, and total variation (TV) smoothness loss $L_{tv}$:
  $$
  L_{tu} = \lambda_3 L_{ide} + \lambda_4 L_{dc} + \lambda_5 L_{tv}
  $$
  with total teacher loss $L_t = L_{ts} + L_{tu}$.

- **Student Supervised Loss:** Employs the uncertainty-guided supervised loss $L_{ugs}$ and the adversarial loss:
  $$
  L_{ss} = \lambda_1 L_{ugs} + \lambda_2 L_a
  $$
- **Student Unsupervised Loss:** Sum of uncertainty-guided consistency loss $L_{ugu}$, dark-channel prior, TV smoothness, and a KL divergence distillation term:
  $$
  L_{su} = \lambda_3 L_{ugu} + \lambda_4 L_{dc} + \lambda_5 L_{tv} + \lambda_6 L_{kl}
  $$
  where
  $$
  L_{kl} = \mathrm{KL}\bigl(\mathrm{Softmax}(V_{\mathrm{real}}),\,\mathrm{Softmax}(V_{\mathrm{syn}})\bigr)
  $$
and total student loss $L_s = L_{ss} + L_{su}$.

The KL distillation term is pivotal for domain adaptation, encouraging the student to align feature distributions between synthetic and real domains.

## 3. Uncertainty Estimation and Aleatoric Modeling

A dedicated Uncertainty Estimation Block (UEB) models pixel-wise aleatoric uncertainty $\theta$. For a synthetic image $I_i$, with dehazed teacher output $G_1(I_i)$ and ground truth $\hat{J}_i$, a Laplace observation model yields:
$$
\hat{J}_i = G_1(I_i) + \epsilon\theta_i,\quad \epsilon\sim \mathrm{Laplace}(0,1)
$$
The corresponding negative log-likelihood loss, up to additive constants, is
$$
-\ln p(\hat{J}_i|I_i) = \frac{\|\hat{J}_i - G_1(I_i)\|_1}{\theta_i} + 2\ln\theta_i
$$
The uncertainty loss becomes
$$
L_{ue} = \frac{1}{N}\sum_{i=1}^N\left[e^{-\ln\theta_i}\|\hat{J}_i - I_i\|_1 + 2\ln\theta_i\right]
$$
Predicted uncertainty modulates both supervised and unsupervised student loss terms, focusing the model on regions with high expected error:
- Supervised: 
  $$
  L_{ugs} = \frac{1}{N} \sum_i \left(\ln\theta_i - \min_k \ln\theta_k\right)\|\hat{J}_i - G_2(I_i)\|_1
  $$
- Unsupervised: 
  $$
  L_{ugu} = \frac{1}{N}\sum_j\left(\ln\theta_j - \min_k\ln\theta_k\right)\|J_j - G_2(J_j)\|_1
  $$

## 4. Training Protocol and Domain Adaptation

The training regime utilizes 10,000 synthetic hazy/clean pairs (RESIDE OTS) and 2,000 unpaired real hazy photographs (URHI). The teacher is trained for 100 epochs with a 5:1 update ratio of supervised to unsupervised steps, using a learning rate decaying linearly from $10^{-4}$. The student, initialized with the teacher’s final weights and with the UEB frozen, trains for 60 epochs with a 5:1 unsupervised to supervised update ratio, following the same learning rate schedule.

Domain adaptation is operationalized via the KL distillation loss $L_{kl}$ on high-level embeddings from the MDB, which compels the student network to transfer feature knowledge learned on synthetic data to unlabelled real inputs.

## 5. Empirical Results and Ablation Analysis

Semi-UFormer achieves state-of-the-art performance on synthetic and real-world benchmarks. On SOTS-Outdoor, it yields PSNR/SSIM scores of 26.87/0.928, and 28.72/0.934 on HSTS—surpassing previous work by an average of over 2 dB. On 50 real images, no-reference metrics (SSEQ ↓, σ-color ↓, HCC ↑) are also optimal or near-optimal: SSEQ=37.75, σ-color=0.0001, HCC=0.2340.

Ablation studies demonstrate cumulative benefits of each component:

| Variant                                 | PSNR      | SSIM   |
|------------------------------------------|-----------|--------|
| Base (DehazeFormer-S + $L_1$)           | 24.74     | 0.905  |
| + Mix DehazeFormer Block (MDB)           | 25.34     | 0.910  |
| + Uncertainty Estimation (UEB, $L_{ugs}$)| 26.17     | 0.919  |
| + KL Distillation (full Semi-UFormer)    | 26.87     | 0.928  |

Key findings:
- The MDB consistently improves performance via long-range context aggregation.
- The uncertainty module yields ∼0.8 dB improvement by concentrating reconstruction on challenging regions.
- KL distillation narrows the synthetic–real domain gap by an additional ∼0.7 dB.

## 6. Comparative Analysis and Significance

Semi-UFormer’s semi-supervised, uncertainty-aware transformer architecture demonstrates that explicit modeling of pixel-wise uncertainty not only provides confidence estimates but also directs the student’s learning focus during domain adaptation. The approach both quantitatively and qualitatively outperforms prior deterministic dehazing networks, producing sharper structure and better color fidelity on real hazy scenes [2210.16057]. The systematic deployment of adversarial, prior-based, and self-supervised (consistency, KL) objectives—alongside uncertainty normalization—accounts for robust generalization across synthetic and real data distributions.

## 7. Conclusion and Future Prospects

Semi-UFormer establishes a new state-of-the-art for image dehazing by uniting semi-supervised knowledge distillation and explicit uncertainty modeling in a transformer backbone. The integration of per-pixel uncertainty not only quantifies model confidence but also enhances dehazing, especially in high-ambiguity regions such as edges and textures. A plausible implication is that uncertainty-guided supervision could generalize to other domains facing synthetic-to-real gaps and spatially localized ambiguity [2210.16057].

Source: https://www.emergentmind.com/topics/semi-uformer