---
title: 'Flickerformer: Burst Flicker Restoration'
url: https://www.emergentmind.com/topics/flickerformer
type: topic
---

# Flickerformer: Burst Flicker Restoration

Searching arXiv for the cited paper and closely related deflickering work to ground the article.
Flickerformer is a transformer-based burst restoration network for removing flicker-band artifacts in short-exposure photography. It is introduced in “It Takes Two: A Duet of Periodicity and Directionality for Burst Flicker Removal” [2603.22794] and is designed around two physical priors of flicker: periodicity, associated with AC-powered illumination, and directionality, associated with rolling-shutter readout. Within this formulation, flicker is treated not as generic noise or uniform underexposure but as a structured spatial-temporal degradation. Flickerformer addresses this structure through three specialized modules: a Phase-based Fusion Module (PFM), an Autocorrelation Feed-Forward Network (AFFN), and a Wavelet-based Directional Attention Module (WDAM) [2603.22794]. Related deflickering research in video has emphasized blind temporal consistency via neural atlases [2303.08120] and histogram-assisted scale-time equalization [2403.06243], but Flickerformer is specifically formulated for burst flicker-band removal under short exposure and rolling shutter.

## 1. Problem setting and artifact model

Flickerformer targets flicker artifacts arising from unstable illumination and row-wise exposure inconsistencies in short-exposure burst photography [2603.22794]. Under AC-powered lighting such as fluorescent and LED sources, illumination intensity oscillates approximately sinusoidally at \(50/60\) Hz, or according to more complex waveforms. Short exposure causes each frame to integrate light over only part of the illumination cycle, while rolling shutter exposes sensor rows sequentially. Their interaction produces both per-frame brightness fluctuations and within-frame striped or banding patterns aligned with the readout direction [2603.22794].

The method is motivated by the claim that these degradations are structured, periodic, and directional rather than spatially homogeneous. Generic restoration models for denoising, low-light enhancement, burst super-resolution, or HDR fusion do not explicitly model these regularities and therefore may preserve banding, attenuate it only partially, or introduce ghosting by mistaking illumination variation for motion [2603.22794]. This places Flickerformer in a distinct methodological category from general blind video deflickering systems, which are formulated for broader temporal inconsistency phenomena, including post-processing or generative artifacts [2303.08120, 2403.06243].

The paper formalizes periodicity in the frequency domain. For a feature map \(\mathbf{X}_t \in \mathbb{R}^{H \times W \times C}\) at frame \(t\), the \(2\)D FFT is written as
\[
\tilde{\mathbf{X}}_t = \mathcal{F}(\mathbf{X}_t) = A_t(\mathbf{k}) e^{i \Phi_t(\mathbf{k})},
\]
where \(A_t(\mathbf{k})\) is magnitude and \(\Phi_t(\mathbf{k})\) is phase [2603.22794]. The article further states that swapping phase spectra across flickering frames changes the spatial distribution of flicker bands, which is used to argue that phase encodes the spatial pattern of flicker [2603.22794].

Directionality is attributed to rolling shutter. If rows are read top to bottom, flicker appears as horizontal bands; if columns are read sequentially, vertical bands result. This means the artifact manifests as directional high-frequency oscillations together with low-frequency dark or bright band regions along a dominant axis [2603.22794]. The model therefore combines frequency-domain periodic analysis with wavelet-domain orientation selectivity.

## 2. Architectural formulation

Flickerformer takes as input a burst of three RGB frames: the base frame \(\mathbf{I}_1\) and neighbors \(\mathbf{I}_0, \mathbf{I}_2 \in \mathbb{R}^{H \times W \times 3}\), and produces a single restored, flicker-free version of the base frame \(\hat{\mathbf{I}}_1\) [2603.22794]. Low-level feature extraction is performed using a \(3\times3\) group convolution, generating frame-wise feature maps \(\mathbf{X}_t\) [2603.22794].

The network is described as an asymmetric U-shaped transformer with four principal stages [2603.22794]. First, PFM aggregates the three frame features into a fused low-level representation \(\mathbf{F}_0\). Second, a three-stage encoder processes the fused representation with increasing channel width and decreasing spatial resolution. Third, a three-stage decoder reconstructs full-resolution features using skip connections from the encoder, with WDAM replacing standard attention in the decoder stages. Fourth, an output head predicts a residual \(\mathbf{R}\) via a \(3\times3\) convolution and adds it to the base frame:
\[
\hat{\mathbf{I}}_1 = \mathbf{I}_1 + \mathbf{R}.
\]
Residual learning is explicitly motivated as a means to focus the network on flicker artifacts rather than full-image content [2603.22794].

At encoder stage \(l\), the spatial resolution is \(\frac{H}{2^l} \times \frac{W}{2^l}\) and the number of channels is \(2^l C\), with multiple transformer blocks per stage [2603.22794]. In these blocks, AFFN replaces a vanilla FFN, thereby embedding periodic-structure modeling directly in the feed-forward pathway. In the decoder, WDAM is used for reconstruction guided by high-frequency directional cues [2603.22794].

The training objective combines \(L_1\) reconstruction loss and VGG-based perceptual loss with equal weights:
\[
\mathcal{L}_{\text{L1}} = \|\hat{\mathbf{I}}_1 - \mathbf{I}_1^{\text{gt}}\|_1,
\]
\[
\mathcal{L}_{\text{perc}} = \sum_j \big\| \phi_j(\hat{\mathbf{I}}_1) - \phi_j(\mathbf{I}_1^{\text{gt}}) \big\|_1,
\]
\[
\mathcal{L} = \mathcal{L}_{\text{L1}} + \mathcal{L}_{\text{perc}}.
\]
Here \(\phi_j\) denotes selected VGG-19 feature maps [2603.22794].

A central distinguishing feature of the architecture is that it reorganizes both fusion and refinement around physical priors rather than generic spatiotemporal aggregation. The paper states that fusion is driven by phase correlation, feed-forward refinement by autocorrelation, and attention by wavelet-domain directionality [2603.22794].

## 3. Periodicity modeling: PFM and AFFN

PFM is the inter-frame periodicity module. It is motivated by the observation that differences between frames arise from both actual scene motion and illumination changes, with the latter largely captured by phase relationships in the frequency domain [2603.22794]. For each input feature \(\mathbf{X}_t\), PFM computes its Fourier representation and extracts the phase. For each reference frame \(t \in \{0,2\}\) relative to the base frame \(1\), it defines a phase similarity map
\[
\mathbf{S}_t(\mathbf{k}) = \Big| e^{i \Phi_t(\mathbf{k})} \odot e^{-i \Phi_1(\mathbf{k})} \Big|,
\]
followed by a learned frequency-domain weighting
\[
\mathbf{W}_t = \sigma(\text{Conv}(\mathbf{S}_t)).
\]
The reference-frame frequency features are then filtered as
\[
\tilde{\mathbf{X}}_t^{\text{enh}} = \tilde{\mathbf{X}}_t \odot \mathbf{W}_t,
\]
mapped back to the spatial domain, and fused with the base-frame feature:
\[
\mathbf{F}_0 = \text{ReLU}\Big(\text{Conv}([\hat{\mathbf{X}}_0, \mathbf{X}_1, \hat{\mathbf{X}}_2])\Big).
\]
This is described as a pre-filter that attenuates flicker-corrupted components before they enter the main encoder-decoder pipeline [2603.22794].

AFFN is the intra-frame periodicity module. It is intended to exploit structural regularities within each scale, replacing pointwise transformer FFNs with a dual-domain refinement pathway [2603.22794]. The module uses the Wiener–Khinchin theorem to compute a \(2\)D spatial autocorrelation for encoder feature \(\mathbf{F}_l\):
\[
\mathbf{R}_l = \mathcal{F}^{-1}\!\left(\mathcal{F}(\mathbf{F}_l) \odot \overline{\mathcal{F}(\mathbf{F}_l)}\right)
= \mathcal{F}^{-1}\!\left(|\mathcal{F}(\mathbf{F}_l)|^2\right).
\]
This operation is said to amplify repeating stripe patterns while suppressing uncorrelated noise [2603.22794].

AFFN then combines spectral power and autocorrelation through
\[
\hat{\mathbf{F}}_k = \mathcal{F}(\mathbf{F}_l) + \alpha \big|\mathcal{F}(\mathbf{F}_l)\big|^2,
\]
\[
\hat{\mathbf{F}}_l = \mathcal{F}^{-1}(\hat{\mathbf{F}}_k) + \beta \mathbf{R}_l,
\]
where \(\alpha\) and \(\beta\) are learnable scalars or vectors [2603.22794]. The resulting features are passed to a gated depthwise FFN:
\[
\mathbf{F}_{\text{out}} = \text{DWConv}\big(\text{GELU}(\hat{\mathbf{F}}_l^1) \odot \hat{\mathbf{F}}_l^2\big),
\]
after channel splitting into two halves [2603.22794]. The paper characterizes this as a periodicity-aware FFN that makes the model more robust to subtle structured banding.

A plausible implication is that PFM and AFFN divide periodic modeling by scale and function: PFM handles inter-frame periodic agreement before deep encoding, whereas AFFN reinforces periodic spatial structure after fusion. That complementary division is consistent with the ablation results reported for individual module replacement [2603.22794].

## 4. Directionality modeling: WDAM

WDAM is the directionality-specific attention module used in the decoder [2603.22794]. It is based on the observation that flicker bands have dominant orientation and are naturally represented in wavelet subbands. For each feature map \(\mathbf{F}_l\), a one-level Haar discrete wavelet transform produces
\[
[\mathbf{F}_{LL}, \mathbf{F}_{LH}, \mathbf{F}_{HL}, \mathbf{F}_{HH}] = \text{DWT}(\mathbf{F}_l),
\]
where \(\mathbf{F}_{LL}\) is the low-frequency approximation, \(\mathbf{F}_{LH}\) and \(\mathbf{F}_{HL}\) are horizontal and vertical high-frequency components, and \(\mathbf{F}_{HH}\) is the diagonal high-frequency component [2603.22794]. The paper states that flicker edges are especially prominent in \(\mathbf{F}_{LH}\) and \(\mathbf{F}_{HL}\).

Attention is applied on the low-frequency branch. Queries, keys, and values are projected from \(\mathbf{F}_{LL}\):
\[
\mathbf{Q} = \mathbf{F}_{LL}\mathbf{W}^Q,\quad
\mathbf{K} = \mathbf{F}_{LL}\mathbf{W}^K,\quad
\mathbf{V} = \mathbf{F}_{LL}\mathbf{W}^V,
\]
with window-based multi-head attention over non-overlapping \(M \times M\) windows [2603.22794]. Directional guidance is extracted from high-frequency subbands by concatenating \([\mathbf{F}_{LH}, \mathbf{F}_{HL}]\) and applying convolution plus sigmoid:
\[
\mathbf{M} = \sigma(\text{Conv}([\mathbf{F}_{LH}, \mathbf{F}_{HL}])).
\]
The attention output is then defined as
\[
\text{Att}(\mathbf{Q}, \mathbf{K}, \mathbf{V}, \mathbf{M}) =
\text{Softmax}\left( \frac{\mathbf{Q}\mathbf{K}^\top}{\sqrt{d}} + \mathbf{B} \right) \big( \mathbf{M} \odot \mathbf{V} \big),
\]
where \(\mathbf{B}\) is a learnable relative position bias [2603.22794]. The key deviation from standard attention is that the value branch is modulated by a directional map derived from oriented wavelet coefficients.

After low-frequency refinement, high-frequency subbands are updated by lightweight convolutions and the final output feature is reconstructed by inverse DWT:
\[
\mathbf{F}_{\text{att}} = \text{IDWT}(\mathbf{F}'_{LL}, \mathbf{F}'_{LH}, \mathbf{F}'_{HL}, \mathbf{F}'_{HH}).
\]
The article also provides an efficiency argument: since attention is applied only on the half-resolution \(LL\) subband, the attention cost is reduced by approximately \(4\times\), with an added \(\mathcal{O}(HWC)\) cost for generating \(\mathbf{M}\) [2603.22794].

This wavelet-guided formulation differs from generic isotropic self-attention by assigning explicit restoration priority to low-frequency dark or bright regions indicated by directional high-frequency edges. In the context of flicker-band removal, this is a localization mechanism rather than merely a compression strategy.

## 5. Training protocol, dataset, and empirical results

Flickerformer is trained and evaluated on BurstDeflicker, described as a benchmark dataset for burst flicker removal in dynamic scenes, containing real-world flicker artifacts under various AC lighting conditions together with flicker-free ground truth [2603.22794]. The paper states that the dataset includes diverse scenes and motions and that the flicker patterns are real rather than purely synthetic [2603.22794].

The reported training configuration is a three-level encoder-decoder with block counts \([2,2,2]\), attention heads \([1,2,4]\), channels \([32,64,96]\), and AFFN expansion factor \(\gamma = 2.66\) [2603.22794]. Optimization uses Adam with initial learning rate \(1 \times 10^{-4}\), and the input protocol uses bursts of three frames [2603.22794]. The text notes that data augmentation details are not specified, while remarking that random cropping and flipping are typical in similar work; that remark should be treated as contextual rather than as a documented implementation detail [2603.22794].

The paper compares Flickerformer with 16 state-of-the-art restoration models repurposed to burst input, spanning HDR reconstruction, burst super-resolution, deblurring, low-light enhancement, and general restoration [2603.22794]. On the BurstDeflicker test set, the reported metrics are as follows:

| Method | PSNR / SSIM / LPIPS | Params / FLOPs |
|---|---:|---:|
| Flickerformer | 31.226 dB / 0.920 / 0.045 | \(\approx 3.92\)M / \(\approx 128.76\)G |
| AST | \(\sim 30.646\) dB / 0.918 / 0.050 | not stated here |
| HINT | lower PSNR/SSIM, LPIPS 0.046 | not stated here |

The paper states that Flickerformer achieves approximately \(+0.58\) dB PSNR improvement over the second-best PSNR method, with better or equal SSIM and lower LPIPS than competing methods, while using significantly fewer parameters than some larger baselines [2603.22794]. Visual comparisons are reported to show more uniform illumination, more faithful color, successful restoration of screens and high-contrast regions without smearing, and reduced ghosting in dynamic scenes [2603.22794].

Ablation results are central to the paper’s argument. Replacing AFFN with vanilla FFN, LeFF, GDFN, or FRFN reduces PSNR, with AFFN improving by approximately \(+0.27\) to \(0.9\) dB over these alternatives [2603.22794]. Replacing WDAM with Swin SA, Top-k SA, Condensed SA, or ASSA decreases PSNR by about \(0.23\) to \(0.33\) dB [2603.22794]. Starting from an AST-like baseline with CNN fusion, FRFN, and ASSA, replacing the corresponding components with PFM, AFFN, and WDAM yields individual gains of \(+0.279\), \(+0.382\), and \(+0.373\) dB PSNR respectively, and combining all three produces the full 31.226 dB result [2603.22794].

## 6. Position within deflickering research

Flickerformer belongs to a narrower problem setting than blind video deflickering frameworks. “Blind Video Deflickering by Neural Filtering with a Flawed Atlas” [2303.08120] addresses temporally inconsistent videos from varied sources, including old films, slow motion, time-lapse, and video processing artifacts, using a neural atlas plus a learned neural filter. That method emphasizes blind operation from a single flickering video and long-range temporal consistency through atlas coordinates rather than burst fusion [2303.08120]. BlazeBVD [2403.06243] instead uses illumination histograms and scale-time equalization to derive deflickering priors, then applies a 2D U-Net, local flow-based restoration, and a lightweight 3D consistency model [2403.06243]. Both works operate primarily in the video domain and are formulated for broader flicker types than rolling-shutter flicker bands.

By contrast, Flickerformer is specialized for short-exposure burst photography under AC lighting and rolling shutter [2603.22794]. Its contribution is not blind deflickering in the general video sense but explicit exploitation of two physical priors that are tightly coupled to the capture mechanism. This specialization explains why the architecture uses phase similarity, autocorrelation, and oriented wavelet attention rather than atlas coordinates or histogram smoothing. The differences are not only architectural but ontological: in Flickerformer, flicker is treated as a burst-local structured degradation with stripe-like regularity, whereas the broader video methods model flicker as temporal inconsistency that may be global, local, long-term, short-term, or algorithm-induced [2303.08120, 2403.06243].

A common misconception would be to regard Flickerformer as a generic deflickering transformer applicable without modification to arbitrary video flicker. The paper does not support that interpretation. Its input protocol is a burst of three frames, its priors are derived from AC-powered illumination periodicity and rolling-shutter directionality, and its evaluation is on BurstDeflicker rather than on general video deflickering benchmarks [2603.22794]. This suggests domain specificity rather than universal deflickering scope.

## 7. Limitations, assumptions, and implications

The paper explicitly notes a coverage assumption: across the burst, different frames should provide clean or less-degraded observations for each spatial region [2603.22794]. If a region remains dark or fully flicker-extinguished in all frames, there is no clean observation to recover, and restoration becomes partial. An example given is an area under a turned-off long strip light that remains poorly reconstructed [2603.22794].

Additional practical limitations are described as implied rather than formally benchmarked. The architecture remains a transformer U-Net with FFT and DWT operations, so despite WDAM’s efficiency relative to full-resolution attention, deployment on embedded or mobile systems may require further optimization [2603.22794]. The model is also tuned to strip-like rolling-shutter flicker, so highly irregular or non-periodic illumination changes may not align with its priors [2603.22794].

The broader literature cited in the source block suggests several plausible extensions. One is longer-burst or video deflickering formulations that preserve Flickerformer’s periodicity and directionality modeling while expanding temporal context [2603.22794]. Another is adaptation to related artifacts such as PWM screen banding or LED display moiré, where periodic and directional structures may also be salient [2603.22794]. A further implication, in light of blind video deflickering approaches [2303.08120, 2403.06243], is that future systems could hybridize explicit physical priors with more general long-range temporal representations.

In summary, Flickerformer is defined by a specific claim: flicker-band artifacts in burst photography should be modeled through periodicity and directionality rather than subsumed under generic restoration. Its three modules instantiate that claim at inter-frame fusion, intra-frame refinement, and decoder attention, respectively. The reported gains on BurstDeflicker, together with targeted ablations, support the view that explicit physical structure is materially beneficial in this restoration regime [2603.22794].

Source: https://www.emergentmind.com/topics/flickerformer