---
title: Two-Phase Uncertainty-Aware Shifted Windows Transformer
url: https://www.emergentmind.com/topics/two-phase-uncertainty-aware-shifted-windows-transformer
type: topic
---

# Two-Phase Uncertainty-Aware Shifted Windows Transformer

The **Two-Phase Uncertainty-Aware Shifted Windows Transformer** denotes a transformer-based smoke-detection framework that combines optical flow-guided feature fusion, a shared Swin Transformer backbone, and explicit predictive-uncertainty estimation under a two-phase training schedule [2508.14597]. Its architectural substrate is the Swin Transformer, a hierarchical vision transformer in which computation alternates between regular window-based self-attention and shifted-window self-attention, thereby preserving local-attention efficiency while enabling cross-window information flow [2103.14030]. In this sense, the designation “two-phase” combines two distinct mechanisms: the Swin backbone’s alternating **W-MSA $\rightarrow$ SW-MSA** computation and the detector’s **Phase I $\rightarrow$ Phase II** optimization regimen for accuracy learning and uncertainty calibration.

## 1. Conceptual definition and scope

TP-UAST was introduced for **reliable smoke detection from monocular RGB video frames**, a safety-critical setting in which smoke exhibits complex spatiotemporal dynamics and can resemble fog, steam, dust, and clouds. The framework is motivated by the need for not only accurate smoke/non-smoke discrimination but also calibrated confidence under illumination variability, turbulent plume motion, and environmental noise. Its pipeline fuses appearance and motion cues, then predicts a class score together with uncertainty-related quantities intended to support risk-aware decisions [2508.14597].

A central point of clarification is that **uncertainty awareness is not part of the original Swin Transformer**. Swin is a deterministic hierarchical backbone with shifted-window self-attention and relative position bias; it does not include probabilistic attention, uncertainty estimation, or explicit aleatoric/epistemic modeling. TP-UAST should therefore be understood as a task-specific extension built on Swin rather than as a property of Swin itself [2103.14030].

Another recurrent source of ambiguity concerns the word “phase.” In TP-UAST, it refers both to the **two-block alternation of regular and shifted windows** inherited from Swin and to the **two-phase learning regimen** used to decouple detection performance from variance learning. These are separate constructs operating at different levels of the system: one architectural, one optimization-related [2508.14597].

## 2. Backbone architecture and shifted-window computation

The backbone is a **shared Swin Transformer backbone** applied independently to two input branches: an RGB image and a segmented color-encoded optical-flow map. Patch embedding uses **non-overlapping $4\times 4$ patches**, a **linear embedding dimension of 96**, and reduces the input resolution by $4\times$. The encoder has four hierarchical stages with patch merging between stages, while the decoder contains **five sequential decoder blocks** that upsample by bilinear interpolation, concatenate skip features from the encoder, and apply **two conv+ReLU layers per block** to restore spatial detail [2508.14597].

| Stage | Blocks / heads / window | Feature dim |
|---|---|---|
| Stage 1 | 2 blocks, 4 heads, $7\times 7$ | 96 |
| Stage 2 | 2 blocks, 4 heads, $7\times 7$ | 192 |
| Stage 3 | 4 blocks, 8 heads, $7\times 7$ | 384 |
| Stage 4 | 2 blocks, 8 heads, $7\times 7$ | 768 |

Within each stage, Swin computes self-attention in **non-overlapping local windows** of size $M\times M$, with $M=7$ by default. In a regular **W-MSA** block, tokens attend only within their own window. In the subsequent **SW-MSA** block, the feature map is cyclically shifted by $\Delta=(\lfloor M/2\rfloor,\lfloor M/2\rfloor)$ before window partitioning, and an attention mask prevents invalid interactions between tokens that originated in different windows before shifting. This produces cross-window communication over successive blocks without reverting to global attention [2508.14597].

The Swin formulation is hierarchical: patch merging concatenates each $2\times 2$ token group, halves spatial resolution, and increases channel width at stage transitions, producing multi-resolution features that can be interpreted at scales such as $H/4$, $H/8$, $H/16$, and $H/32$. The original Swin paper characterizes this design as having **linear computational complexity with respect to image size**, in contrast to global MHSA, and reports strong backbone performance across classification and dense prediction tasks, including **87.3 top-1 accuracy on ImageNet-1K**, **58.7 box AP and 51.1 mask AP on COCO test-dev**, and **53.5 mIoU on ADE20K val** for Swin-L configurations [2103.14030].

## 3. Optical-flow-guided feature fusion

Before transformer inference, TP-UAST constructs an explicit motion representation using a **four-color-theorem-inspired dual-phase level-set fractional-order variational model (FCDLe-FOV)** for dense optical flow. This model uses an $L1$ data term, Marchaud fractional derivatives of order $\alpha\in(0,1)$, and dual-phase level-set boundary regularization. The reported purpose is to **preserve motion discontinuities** and maintain boundary-aware segmentation of moving smoke structures [2508.14597].

The optical flow is then **color-encoded** and segmented into smoke versus background by a **Gaussian Mixture Model (GMM)**. A binary mask suppresses non-smoke motion, and the segmented motion map is obtained by multiplying that mask with the color-encoded optical-flow image. The resulting paired inputs to TP-UAST are therefore two three-channel tensors:

- **Appearance branch**: RGB image  
- **Motion branch**: segmented color-encoded optical-flow map

The article’s technical description states that the two branches pass through the **same backbone-decoder with shared weights**. The uncertainty-aware head consumes the final feature maps from each branch in parallel. The paper does **not** state an explicit cross-branch concatenation mechanism; instead, it specifies that the head operates on final features from each branch and that random transformations used for Monte Carlo epistemic sampling are shared across both inputs to preserve correspondence [2508.14597].

The GMM stage is described as using a **$K$-component GMM** over pixel intensities or color vectors, with **$K$ selected empirically**; the paper does not state the value of $K$ explicitly. The resulting dataset pairs RGB frames with segmented optical-flow maps and binary masks, is released on **Kaggle**, and uses images resized to **$256\times 256$** for TP-UAST. The paper does not report dataset size, class balance, or train/validation/test split details, but states that experiments cover diverse scenes including forests, roads, industrial settings, and crowds under stationary and non-stationary illumination [2508.14597].

## 4. Two-phase learning and uncertainty modeling

The prediction module is uncertainty-aware at the head level. For each branch’s final feature map, the architecture applies **global average pooling**, then **FC with dropout**, followed by **three parallel FC heads** producing a **primary classification output $p$**, a **mean output $\mu$**, and a **variance output $\sigma^2$**, with the variance constrained to be non-negative by an activation that is not explicitly specified in the paper [2508.14597].

The training schedule is explicitly divided into two phases. **Phase I** optimizes the classification head and mean-prediction head while the variance head is frozen. Supervision uses a composite **binary cross-entropy** objective on both $p$ and $\mu$. The reported optimizer is **Adam**, with **initial learning rate $1\times 10^{-4}$** and **polynomial decay**, and training continues until the **epoch-averaged loss $< 0.2$**. **Phase II** freezes all weights except the variance sub-head, estimates epistemic variability by Monte Carlo sampling under random flips and rotations with a shared seed across the paired inputs, computes empirical mean and variance from the sampled predictions, and trains the variance head using an **$L1$ loss** against the Monte Carlo variance estimate [2508.14597].

The uncertainty model decomposes predictive uncertainty into aleatoric and epistemic components:
$$
\mathrm{Var}[y \mid x] =
\underbrace{\mathrm{Var}_{\mathrm{aleatoric}}}_{\text{data noise}} +
\underbrace{\mathrm{Var}_{\mathrm{epistemic}}}_{\text{model uncertainty}}.
$$

In the reported implementation, **aleatoric uncertainty** is learned directly through the variance head output $\sigma^2$, whereas **epistemic uncertainty** is approximated by the variance induced by stochastic test-time transformations. The paper explicitly notes that it does **not** use Bayesian weight posteriors, ensembles, or MC dropout; epistemic uncertainty arises from **test-time augmentation** rather than posterior inference over model parameters [2508.14597].

At inference time, the paper reports both **PP $\equiv p$** and a **plausibility confidence**
$$
C = \exp\!\left(-\tfrac{1}{2}Z^2\right),
$$
with
$$
Z = (\ell - \mu)/\sigma,
$$
where $\ell$ is the logit or score draw under the Gaussian interpretation used for plausibility visualization. This construction is used to flag ambiguous cases via low-confidence tails in the plausibility distribution [2508.14597].

## 5. Reported empirical behavior

The paper reports results at two coupled levels: the **optical-flow front end** and the **TP-UAST detector** itself. For optical flow, FCDLe-FOV is stated to outperform **NFVLS, FS-FOV, NLW, HS-NE** on **Middlebury1, Middlebury2, and Sintel1** with respect to **AAE, AEPE, and AENG**. Example values given are **Middlebury1: AAE=0.173, AEPE=0.877, AENG=1.875**; **Middlebury2: AAE=0.176, AEPE=2.330, AENG=2.461**; and **Sintel1: AAE=0.101, AEPE=0.393, AENG=1.079** [2508.14597].

Under noise perturbations, the paper reports **SSIM on color maps** for Gaussian, Poisson, and salt-and-pepper noise. Example values include **Smoke1: 0.87, 0.95, 0.97**; **Smoke4: 0.91, 0.97, 0.98**; **Non-Smoke1: 0.97, 0.98, 0.99**; and **Non-Smoke4: 0.95, 0.98, 0.99**. These figures are presented as evidence that the color-encoded flow representation remains stable under multiple noise types [2508.14597].

For smoke detection, the reported **validation-set** discriminative metrics are **Accuracy 1.00, Precision 1.00, Recall 1.00, and F1 1.00**, and the confusion matrix is said to show **zero false positives and zero false negatives**. The strongest listed competitors remain below TP-UAST: **MFFNet** with **Accuracy 0.98, Precision 0.94**; **Safarov et al.** with **Precision 0.96, Recall 0.97**; an **energy-efficient model** with **Accuracy/Precision/Recall/F1 all 0.95**; and **YOLOv2** with **Accuracy 0.96, Precision 0.97, Recall 0.95, F1 0.97** [2508.14597].

The uncertainty outputs are evaluated by several calibration-oriented diagnostics. The paper reports a **reliability diagram with 10 bins**, a calibration curve **nearly on the diagonal**, and **Expected Calibration Error (ECE) $\approx 0$**. The uncertainty histogram is described as **sharply peaked at $<0.005$**, with a tail extending to **$\approx 0.06$**; larger predicted $\sigma$ correlates with larger absolute error; both classes have **median $\sigma \approx 0.002$**, but the smoke class exhibits heavier upper quartile and outliers. The standardized score distribution shows **$\approx 68\%$ within $|Z|\le 1$** and **$\approx 95\%$ within $|Z|\le 2$**, while the plausibility confidence satisfies **$>80\%$ samples with $C>0.8$** [2508.14597].

The paper does **not** report explicit ablations isolating shifted versus non-shifted windows, two-phase versus single-phase training, or motion fusion versus appearance-only inputs. It nevertheless states three qualitative conclusions: shifted windows are expected, following Swin, to improve cross-window communication without global attention cost; two-phase training **demonstrably improves calibration** with **ECE $\approx 0$** and well-behaved $Z/C$ distributions; and optical-flow fusion qualitatively suppresses background motion while emphasizing smoke regions [2508.14597].

## 6. Limitations, applications, and relation to broader transformer research

The primary application domain is **reliable early smoke detection** in safety-critical scenarios. The paper identifies three use-case categories: **surveillance and early fire detection in public spaces**, **industrial safety monitoring** such as factories and refineries, and **autonomous environmental monitoring** in forests and construction sites [2508.14597].

Its limitations are correspondingly explicit. The method relies on **monocular imagery** and does not use depth or multispectral cues. It may fail on **thin, wispy smoke under complex sky/background or low-light clutter**. It also provides **no reported real-time throughput**, leaving embedded deployment constraints unresolved. Future work is directed toward **real-time embedded implementation on limited-compute platforms** and **multi-class hazard detection** involving fire, steam, and dust together with richer multimodal fusion [2508.14597].

In relation to the broader transformer literature, TP-UAST inherits the key advantages that made Swin a general-purpose vision backbone: hierarchical feature construction, local self-attention with shifted-window cross-window interaction, and computational scaling that is substantially smaller than global MHSA. A common misconception is that the “uncertainty-aware shifted windows transformer” is simply Swin under a different name. More precisely, TP-UAST is a **domain-specific extension of Swin** in which the backbone remains a shifted-window transformer, while uncertainty awareness is introduced through an additional prediction head, Monte Carlo test-time augmentation, and a two-phase optimization schedule [2103.14030].

Source: https://www.emergentmind.com/topics/two-phase-uncertainty-aware-shifted-windows-transformer