PBAN: Bidirectional Attention for SR IQA

• The paper introduces a deep full-reference model for super-resolved image quality assessment that aligns HR and SR features via bidirectional attention.
• It employs advanced modules like GMDC and SubEC to capture distortion-aware, multi-scale features and refine perceptual quality predictions.
• Empirical results on QADS, CVIU, and Waterloo show that PBAN outperforms traditional metrics by effectively detecting both fidelity degradation and enhancement-induced distortions.

Perception-oriented Bidirectional Attention Network (PBAN) is a deep full-reference image quality assessment model designed specifically for super-resolved images. In the SR FR-IQA setting, it takes an original high-resolution reference image and a super-resolved image produced by an SR algorithm, and predicts a perceptual quality score intended to correlate with human subjective ratings such as MOS. Its defining premise is that SR quality assessment must address not only fidelity loss but also enhancement-induced distortions—such as hallucinated textures, over-sharpened edges, and structurally plausible but incorrect details—and that this requires explicit bidirectional interaction between reference and SR feature spaces rather than generic similarity measures alone (Li et al., 8 Sep 2025). Subsequent work extended the idea into an image-level perceptual predictor, Efficient-PBAN, and used it as a differentiable perceptual loss for SR optimization (Zhou et al., 25 Feb 2026).

1. Problem formulation and perceptual motivation

PBAN was proposed for full-reference image super-resolution quality assessment. The task is to compare an SR image against its HR reference and regress a perceptual quality score. The paper distinguishes this setting from standard FR-IQA by emphasizing that SR outputs contain unusual distortions: visually appealing but incorrect textures, edge ringing, oversharpening, and fabricated details. Accordingly, the evaluator must judge not only fidelity degradation but also the perceptual plausibility and correctness of enhancement (Li et al., 8 Sep 2025).

The model is motivated by three human-visual-system properties explicitly named in the paper: attention to distortions, spatial frequency sensitivity, and hierarchical perception. SR quality assessment is framed as involving two perceptual processes. The first is generate distortion, in which SR generation from HR/LR introduces distortions. The second is evaluate distortion, in which a human observer compares the SR image against the HR reference or an internal expected scene. This motivates a bidirectional alignment strategy: HR features should be aligned toward SR space to reflect distortion generation, while SR features should be aligned toward HR space to reflect distortion evaluation.

A central criticism of earlier SR-IQA practice is that traditional FR-IQA metrics such as PSNR, MSE, SSIM, and MS-SSIM primarily target low-level differences or generic structural similarity and were developed for common degradations such as blur, noise, and compression. Even deep FR-IQA methods are characterized as being trained largely around degradative distortions, whereas SR introduces enhancement-induced distortions. PBAN is therefore positioned as a deep, full-reference, SR-specific quality predictor rather than a generic FR-IQA metric.

2. Architectural organization

PBAN consists of three modules: Image Encoder Module, Perception-oriented Bidirectional Attention (PBA) Module, and Quality Prediction Module (Li et al., 8 Sep 2025).

The end-to-end pipeline is patch based. The HR reference image and SR image are first divided into non-overlapping $32\times32$ patches. Each HR patch and corresponding SR patch is independently encoded by an image encoder composed of a convolutional layer with $3\times3$ kernels, batch normalization, and ReLU. The encoded features are denoted

$$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$

These branch features are then passed through one or more stacked PBA blocks. Each PBA block contains three components:

• GMDC: Grouped Multi-scale Deformable Convolution
• Bi-Atten: Bidirectional Attention
• SubEC: Sub-information Excitation Convolution

After the PBA stage, the two refined branch outputs $O^{HR}_{PBA}$ and $O^{SR}_{PBA}$ are average-pooled, flattened, mapped by fully connected layers, concatenated, and regressed to a scalar quality score. In the quality prediction module, linear layers reduce the feature dimension gradually from 1024 to 1. Patch-level predictions are finally averaged to produce the image-level quality estimate.

This architecture is explicitly two-branch and full-reference. One branch processes HR-derived features, and the other processes SR-derived features. The design choice is consequential: PBAN does not treat quality assessment as independent feature extraction followed by late fusion, but as feature interaction between reference and SR representations inside the core attention block.

3. Perception-oriented bidirectional attention

The PBA module is the central mechanism in PBAN. It is composed of Bi-Atten, with GMDC placed before attention score construction and SubEC after attention to refine distortion-aware features (Li et al., 8 Sep 2025).

“Bidirectional” here means that attention is constructed in both directions between HR and SR feature spaces. The paper defines:

• HR $\rightarrow$ SR: the HR branch uses SR-derived key features to align HR features toward SR space.
• SR $\rightarrow$ HR: the SR branch uses HR-derived key features to align SR features toward HR space.

This differs from one-way cross-attention. The two directions are associated with the paper’s perceptual interpretation of distortion generation and distortion evaluation.

For feature projection, PBAN uses $3\times3$ convolutions: $$\begin{aligned} &Q^{HR},K^{HR},V^{HR}=Conv(X^{HR}), \ &Q^{SR},K^{SR},V^{SR}=Conv(X^{SR}). \end{aligned}$$

A key technical distinction from generic cross-attention is the paper’s preference for $Q$-$3\times3$0 homology rather than $3\times3$1-$3\times3$2 homology. In PBAN, query and value come from the same branch, while key comes from the other branch. The stated rationale is that HR and SR images are strongly related, and the goal is to emphasize subtle distortions while preserving the target branch’s feature semantics.

Before attention computation, the keys are processed by GMDC: $3\times3$3

The bidirectional attention is then defined using HR queries and values with SR-derived keys for one branch, and symmetrically for the other. Although the paper’s typesetting of the softmax formula is corrupted in the accessible text, the construction is explicit: the attention logits are normalized by variance terms $3\times3$4 and $3\times3$5, and the key is always cross-branch while query and value remain same-branch.

The authors contrast this design with earlier attention-based SR-IQA methods such as JCSAN and TADSRNet. PBAN is described as being full-reference, as avoiding pooling-based spatial/channel attention, as explicitly aligning HR and SR feature spaces, and as focusing attention on distortion discrepancies rather than generic saliency. The attended content is therefore not semantic object saliency in the ordinary sense, but distortion-sensitive HR/SR correspondences, including false textures, oversharpening, local structural inconsistencies, and texture or edge artifacts.

4. Distortion-aware refinement: GMDC and SubEC

PBAN adds two auxiliary mechanisms around bidirectional attention: Grouped Multi-scale Deformable Convolution (GMDC) and Sub-information Excitation Convolution (SubEC) (Li et al., 8 Sep 2025).

GMDC is introduced to let the model adaptively perceive distortion regions before attention is computed. The paper links this to HVS spatial frequency sensitivity and argues that SR distortions are local, scale-varying, structurally irregular, and often concentrated around edges and textures. Standard convolution is written as

$3\times3$6

while deformable convolution adds learnable offsets: $3\times3$7

GMDC splits channels into groups, applies an independent deformable convolution to each group with a different kernel size $3\times3$8, and then fuses grouped outputs with a final point-wise $3\times3$9 convolution. The grouped multi-scale structure is presented as the crucial design point; the paper explicitly notes that standard DCN was often worse than the baseline, whereas GMDC consistently improved performance. On CVIU, the best overall configuration with SubEC was GMDC $$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$0 with 2 groups, achieving SRCC 0.978, KRCC 0.872, PLCC 0.981, and RMSE 0.397.

SubEC is applied after bidirectional attention and is meant to refine the resulting distortion-sensitive features by directing perception to both sub-pixel spatial information and sub-channel information. It has three branches: a Sub-Channel Weight branch, a Sub-Pixel Weight branch, and an Identity shortcut branch. Its output is

$$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$1

followed by residual addition: $$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$2

The Sub-Pixel branch reduces channels from $$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$3 to $$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$4 using two $$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$5 convolutions, applies sub-pixel convolution / pixel shuffle with upscaling factor $$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$6, produces an upsampled map, and downsamples it back to obtain Sub-Pixel Weight. The Sub-Channel branch uses adaptive average pooling, grouped $$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$7 expansion, channel shuffle, and $$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$8 compression to obtain Sub-Channel Weight.

The ablations in the paper treat several design choices as consequential. Channel shuffle helps; replacing multiplicative fusion with element-wise addition is worse; sub-pixel convolution performs better than transposed convolution, bilinear, and bicubic; and $$X^{HR}, X^{SR} \in \mathbb{R}^{C \times 32 \times 32}.$$9 is selected as a strong and efficient default. A notable result on QADS is that adding SubEC to Bi-Atten improves performance from SRCC 0.981, PLCC 0.982, RMSE 0.055 to SRCC 0.984, PLCC 0.986, RMSE 0.050.

5. Training protocol and empirical results

PBAN is evaluated on three SR-IQA datasets—QADS, CVIU, and Waterloo—using SRCC, KRCC, PLCC, and RMSE, with a five-parameter nonlinear fitting applied before PLCC and RMSE calculation (Li et al., 8 Sep 2025). Each dataset is randomly split into 80% train and 20% test, all DNN-based methods are trained independently on each dataset, and training uses 5-fold cross-validation on the training set for 100 epochs per fold. The optimizer is SGD with initial learning rate 0.01, momentum 0.9, and weight decay $O^{HR}_{PBA}$0. The implementation uses PyTorch on an NVIDIA DGX Station, Tesla V100-DGXS-32GB GPU under Ubuntu 18.04 LTS.

Patch extraction is central to the training protocol. Each image pair is decomposed into non-overlapping $O^{HR}_{PBA}$1 patches, each patch inherits the MOS of its source SR image, and image-level prediction is obtained by averaging patch scores. The reported numbers of training patch pairs are 129,360 for QADS, 194,400 for CVIU, and 55,800 for Waterloo.

Dataset	PBAN performance	Comparator note
QADS	SRCC 0.986, KRCC 0.923, PLCC 0.987, RMSE 0.044	SRIF: SRCC 0.916, PLCC 0.917, RMSE 0.109
CVIU	SRCC 0.978, KRCC 0.872, PLCC 0.981, RMSE 0.397	TOPIQ: SRCC 0.970, PLCC 0.971, RMSE 0.486
Waterloo	SRCC 0.965, KRCC 0.848, PLCC 0.979, RMSE 0.587	best among all compared methods

These results are interpreted in the paper as evidence that deep SR-aware FR modeling is superior to handcrafted FR SR-IQA, that explicit HR/SR feature interaction is more effective than generic attention or generic FR-IQA metrics, and that PBAN is especially strong at detecting SR-specific enhanced distortions.

Cross-dataset tests are also reported. Training on CVIU and testing on QADS yields SRCC 0.890, KRCC 0.732, and PLCC 0.892, which the paper describes as the best among compared methods. Training on QADS and testing on CVIU gives SRCC 0.730, KRCC 0.550, and PLCC 0.794, with second-best SRCC but best KRCC and PLCC.

The ablations decompose the contributions of the three main components. On QADS and CVIU, a baseline without Bi-Atten, GMDC, and SubEC achieves QADS: SRCC 0.936, PLCC 0.937, RMSE 0.103 and CVIU: SRCC 0.942, PLCC 0.954, RMSE 0.815, while full PBAN reaches QADS: 0.986 / 0.987 / 0.044 and CVIU: 0.978 / 0.981 / 0.397. A Bi-Atten-only ablation further shows that both directions are useful: on QADS, HR$O^{HR}_{PBA}$2SR only gives SRCC 0.957, SR$O^{HR}_{PBA}$3HR only gives 0.955, and bidirectional gives 0.981.

The paper also reports efficiency on QADS: 2.222M parameters, 26.278G FLOPs, and 16.995 ms speed. It characterizes this as the second-lowest parameter count, relatively low FLOPs, and much better accuracy than larger models.

6. Efficient-PBAN and closed-loop perceptual optimization

Efficient-PBAN is an Efficient Perceptual Bi-directional Attention Network proposed as an image-level adaptation and practical successor to PBAN for perceptual quality optimization of image super-resolution (Zhou et al., 25 Feb 2026). Its target is not only quality prediction but also the use of the learned quality predictor as a differentiable perceptual loss inside SR training.

Efficient-PBAN is a full-reference SR quality predictor operating on an image pair

$O^{HR}_{PBA}$4

Both inputs are processed by a ResNet stem and the first residual block (Layer1) with shared parameters; beyond Layer1, the branches are separated to capture distinct SR and HR statistics. The central block is PBA$O^{HR}_{PBA}$5, which produces branch-specific query, key, and value embeddings,

$O^{HR}_{PBA}$6

and computes attention along both the height and width axes. The logits are normalized by their standard deviation rather than by $O^{HR}_{PBA}$7, and the axial outputs are averaged to form directional interaction outputs $O^{HR}_{PBA}$8 and $O^{HR}_{PBA}$9. These are fused by a SubEC module and then passed to a global quality prediction head with average pooling, flatten, linear layers, dropout, ReLU, and concatenation.

A central practical change relative to PBAN is that Efficient-PBAN is image-level rather than patch-level. The paper explicitly motivates this by noting that earlier quality predictors such as PFIQA and PBAN are patch based, making them expensive and awkward to use as end-to-end losses.

The model is trained on a self-constructed SR quality database containing 720 SR images at approximately 2K resolution, generated from 19 high-resolution reference images selected from DIV2K. The database spans four upscaling factors—$O^{SR}_{PBA}$0, $O^{SR}_{PBA}$1, $O^{SR}_{PBA}$2, and $O^{SR}_{PBA}$3—and includes SR outputs from 4 GAN-based, 5 diffusion-based, 4 transformer-based, 1 flow-based, and 1 CNN-based methods. The split is 576 training, 72 validation, and 72 testing samples. Subjective scores were collected in a single-stimulus experiment under ITU-R BT.500-14 conditions with 23 participants aged 18–50 from five regions.

As a predictor, Efficient-PBAN is trained by direct score regression: $O^{SR}_{PBA}$4 After pre-training, it is frozen and inserted into SR training. The paper defines

$O^{SR}_{PBA}$5

and the malformed typesetting in the accessible text suggests an intended weighted combination with coefficients $O^{SR}_{PBA}$6 and $O^{SR}_{PBA}$7. A plausible implication is a weighted objective of the form $O^{SR}_{PBA}$8, which is consistent with the reported setting $O^{SR}_{PBA}$9SSIM + Efficient-PBAN ($\rightarrow$0).

The reported findings follow a consistent pattern. Efficient-PBAN loss improves perceptual quality, SSIM improves fidelity, and SSIM + Efficient-PBAN gives the best overall trade-off. For CAMixerSR on B100, the four settings are:

• Original: PSNR 26.3513, SSIM 0.7188, LPIPS 0.3382, PFIQA 3.8528, EPBAN 2.8682
• +SSIM: PSNR 25.6755, SSIM 0.7304, LPIPS 0.3358, PFIQA 3.8702, EPBAN 2.9415
• +EPBAN: PSNR 25.6203, SSIM 0.6963, LPIPS 0.3567, PFIQA 3.8911, EPBAN 3.3165
• +SSIM+EPBAN: PSNR 25.6741, SSIM 0.7110, LPIPS 0.3560, PFIQA 3.9348, EPBAN 3.4025

A subjective test corroborates the trade-off. For CAMixerSR, MOS rankings are

$\rightarrow$1

while for LINF they are

$\rightarrow$2

The paper also notes a failure mode: Efficient-PBAN-only optimization can produce window artifacts in some patch-based SR backbones, which is why the joint objective with SSIM performs best perceptually.

7. Terminological boundaries, related architectures, and limitations

PBAN should not be conflated with earlier or adjacent bidirectional-attention architectures. Several works are strongly relevant conceptually but are not literally PBAN.

BRIMs—Bidirectional Recurrent Independent Mechanisms—addresses the combination of bottom-up sensory evidence and top-down contextual or prior information using attention and modular recurrent computation. It uses bidirectional inter-layer attention over a null source, the current lower layer, and the previous higher layer, with sparse module activation and within-layer communication. For a PBAN-like reading, its relevance lies in selective routing between bottom-up and top-down streams under modular computation, but it is a recurrent architecture for sequential perception, language, video prediction, and reinforcement learning rather than an SR-IQA network (Mittal et al., 2020).

BFAN, the Bidirectional Focal Attention Network, is an image-text matching model that applies focal attention in both text-to-image and image-to-text directions. Its distinctive claim is that semantic misalignment persists when irrelevant fragments receive nonzero soft attention, so it introduces deterministic masking and renormalization over retained fragments. It is bidirectional and relevance-focused, but its problem setting is cross-modal retrieval rather than SR quality assessment (Liu et al., 2019).

HBFP-Net, the Hierarchical Bi-Directional Feature Perception Network, is a person re-identification model that uses cross-level correlation maps and a Bi-directional Feature Perception module to let low-, middle-, and high-level features reinforce one another. It is closely related in spirit to a perception-oriented bidirectional architecture, but its mechanism is cross-level feature perception for Re-ID rather than HR/SR bidirectional attention for SR FR-IQA (Liu et al., 2020).

Within SR itself, a recurrent source of confusion is the relation between PBAN and Efficient-PBAN. The latter is explicitly described as an efficient image-level extension/adaptation of the earlier PBAN idea rather than a synonym for the 2025 patch-based model (Zhou et al., 25 Feb 2026). The distinction matters because the original PBAN is a patch-level FR-IQA model, whereas Efficient-PBAN is designed to avoid extensive patch sampling and to act as a practical differentiable loss inside SR training.

The limitations are correspondingly task specific. For PBAN, the paper’s explicit future directions are to build a larger-scale SR quality database including newer GAN-based and diffusion-based SR methods, to extend toward video SR quality assessment, and to develop quality-driven SR algorithms (Li et al., 8 Sep 2025). A plausible limitation is that patch-based training with patch MOS inheritance may ignore long-range contextual quality effects. For Efficient-PBAN, the explicit or implied limitations are that it is full-reference, the perception-distortion trade-off remains, perceptual-only optimization can be unstable, and the training set—though broader than earlier SR-IQA databases—still reflects a limited number of references and SR generation pipelines (Zhou et al., 25 Feb 2026).

Taken together, PBAN denotes a specific SR FR-IQA architecture centered on reference-guided distortion alignment through bidirectional attention, while also naming a broader design lineage in which bidirectional feature interaction is used to model perception-sensitive discrepancies rather than generic similarity alone.