Pseudo-Image Data (PID)

• Pseudo-Image Data is algorithmically synthesized image data that mimics unavailable or misaligned ground-truth to support robust training and evaluation.
• PID generation pipelines leverage domain and geometric alignment, along with contrast encoding, to create high-fidelity surrogates with measurable accuracy.
• PID methods enhance image restoration, medical augmentation, and digital preservation by compensating for degraded sensors, limited annotations, and long-term interpretability challenges.

Pseudo-Image Data (PID) denotes a wide class of computationally synthesized image data serving as surrogates for unavailable or imperfectly aligned real images. PID solutions arise across computer vision, medical imaging, and digital archiving tasks, ranging from the synthesis of aligned ground-truth for degraded sensor data, to the generation of pseudo-normal/abnormal variants in pathology, to robust visual encodings for longterm preservation and self-annotation. All PID approaches share the goal of supplying informative, high-fidelity images that mimic (but do not copy) unavailable ground-truth, enabling training, validation, or interpretation under real-world constraints.

1. Fundamental Concepts

Pseudo-Image Data serves as an umbrella term for any image produced by algorithmic manipulation of a related input image, intended to fill the role of unavailable or imperfectly paired "truth" data. This includes, but is not limited to, pseudo-ground-truth for ill-posed restoration, synthetic normality/abnormality in medical imaging, and hybrid analog-digital encodings embedding semantic annotation in the image itself. The defining feature of PID is its algorithmic construction specifically to support downstream use cases—such as supervised restoration or augmentation—under hardware, annotation, or domain constraints for which direct data is costly or unattainable.

2. Generation Pipelines and Architectural Principles

2.1 Aligned PID for Camera Restoration

In under-display camera (UDC) restoration, aligned pseudo-ground-truth is generated from non-aligned stereo pairs, comprising the degraded UDC image $I_D\in\mathbb{R}^{H\times W\times 3}$ and a high-quality reference $I_R\in\mathbb{R}^{H\times W\times 3}$. The following sequential modules are used (Feng et al., 2023):

• Domain Alignment Module (DAM): Transforms $I_D$'s global appearance (color, contrast) to mimic $I_R$, without disturbing the geometric structure. The transformation uses guidance and matching subnets, with AdaIN-based modulation governed by learned affine mappings:

$$\mathrm{AdaIN}(\mathbf x, \mathbf y) = \mathbf y_s \frac{\mathbf x - \mu(\mathbf x)}{\sigma(\mathbf x)} + \mathbf y_b,$$

and contextual loss

$$\mathcal{L}_{\mathrm{DAM}} = \mathcal{L}_{CX}(\hat{I}_D, I_R) = \frac{1}{N} \sum_j \min_i \mathbb{D}(\phi(\hat I_D)_j, \phi(I_R)_i).$$

• Geometric Alignment Module (GAM): Aligns spatial content by learning dense flow-guided correspondences. An off-the-shelf optical flow module (e.g. RAFT) predicts per-pixel shifts, and a flow-guided Transformer restricts self-attention to local neighborhoods, copying high-frequency detail from $I_R$:

$$f_{attn}(\mathbf q_{\mathbf p}) = \sum_{i \in \mathcal N_r(\mathbf p')} \mathrm{softmax}_i\left( \frac{\mathbf q_{\mathbf p}^\top \mathbf k_i}{\sqrt{d}} \right)\mathbf v_i,$$

with stacking at multiple scales. Contextual loss again supervises the final output:

$$\mathcal{L}_{\mathrm{Align}} = \mathcal{L}_{CX}(I_P, I_R).$$

This produces pseudo-supervision pairs $(I_D, I_P)$ where $I_P$ is well-aligned, containing detail from $I_R$ but matching $I_D$'s geometry, suitable for robust restoration model training (Feng et al., 2023).

2.2. PID in Medical Image Synthesis

For pseudo-normality or abnormality synthesis, e.g. lesion removal or insertion, the semi-supervised SMILE framework learns three-way mappings (Du et al., 2021):

• Pseudo-normal image $\hat x = G(x)$ with $G$ a U-Net generator.
• Reconstructive mapping $R$ that uses pseudo-normal $\hat x$ and binary mask $M$ to reconstruct the original abnormal image.
• A U-Net segmentor $S$ enforcing adversarial and cross-entropy constraints:

$$L_{\text{gen}}(G,S) = \| (1-M)\odot I - (1-M)\odot G(I) \|_2^2 + \lambda_{\text{adv}}\;\mathrm{CE}(S(G(I)), M).$$

Unlabeled data is leveraged via confident pseudo-label extraction, closing the learning loop and reducing required annotation.

2.3. Self-Rendering and Hybrid Analog/Digital PID

Digital preservation introduces contrast encoding, mapping $b$-bit grayscale images into binary arrays with blockwise Hamming weights kinduced analog renders. Each original pixel is encoded as a $s\times s$ block ($s=\sqrt{b}$), distributing its information such that the block's average intensity provides an analog preview, while the underlying bits guarantee exact digital recovery (Ruderman, 2011).

3. Canonical PID Algorithms and Their Mathematical Formulations

Application Domain	Key PID Algorithm(s)	Loss/Objective Functions
UDC image restoration	AlignFormer (DAM, GAM)	$\mathcal{L}_\text{DAM}$, $\mathcal{L}_\text{Align}$
Medical pseudo-normality	SMILE (GAN, U-Net, adversarial)	$L_\text{gen}$, $L_\text{recons}$, $L_\text{seg}$
Digital image archiving	Contrast Encoding (block rendering)	Quantization error, MSE, PSNR

Each domain utilizes distinct formulations adapted to its pseudo-supervision task, yet all share a reliance on mapping from an observed, unannotated or degraded image to a PID valid as ground truth for downstream models.

4. Quantitative Benchmarks and Empirical Outcomes

In UDC restoration, AlignFormer-trained PPM-UNet achieves PSNR 22.95 dB, SSIM 0.8581, and LPIPS 0.1236 on held-out pairs, outperforming naive alignment, synthetic data, and prior UDC baselines by substantial margins (e.g., CoBi-trained: PSNR 21.57, SSIM 0.8319, LPIPS 0.1252). Keypoint alignment accuracy (PCK, using LoFTR) is 59% at $\alpha=0.01$, up to 95.1% at $\alpha=0.03$. Ablations confirm the significance of flow-guided attention (RAFT improves PCK by 25%), DAM (alignment drop by 2–3% if removed), and pyramid pooling (PSNR gain of 0.27 dB). Qualitative results show superior texture, color, and subpixel alignment over GAN translation and pixel-wise $\ell_1$ (Feng et al., 2023).

In medical imaging, SMILE surpasses prior GANs (VA-GAN, ANT-GAN, PHS-GAN, GVS) with identity/healthiness scores of 1.00/0.822 (100% supervised) and 0.987/0.810 (75% labels). Data augmentation using PID yields up to +13.9% Dice improvement over no augmentation, and +6% relative to the best single-mode baseline (Du et al., 2021).

Contrast encoding achieves a Pearson correlation of 0.96 (for $b=9$) between grayscale value and Hamming weight; analog preview granularity is limited to $b+1$ levels, with monotonicity and error quantified via:

$$MSE = \frac{1}{WH} \sum_{i=1}^{WH} \epsilon_i^2, \qquad PSNR = 10 \cdot \log_{10} \left( \frac{(2^b-1)^2}{MSE} \right).$$

(Ruderman, 2011).

5. Practical Implications and Application Scenarios

PID methods enable model training and validation in regimes where pixel-perfect labels or ground-truth are infeasible. Notable applications include:

• Image restoration: AlignFormer data unlocks robust learning for UDCs and similar hardware, generalizing to cross-domain super-resolution and ISP design where only stereo or non-aligned reference exists (Feng et al., 2023).
• Medical imaging: PID augments scarce annotated datasets, provides lesion localization via image differencing, and improves model performance in segmentation/detection tasks. SMILE achieves state-of-the-art augmentation gains while reducing dependence on large-scale annotation (Du et al., 2021).
• Digital preservation: Contrast encoding ensures that digital images remain interpretable as analog visualizations far into the future, embedding interpretability in the data arrangement independent of external metadata or software (Ruderman, 2011).

PID generation strategies are broadly extensible to scenarios involving degraded data and reference imagery with substantial spatial or domain misalignment.

6. Limitations, Generalization, and Outlook

Each PID approach reflects assumptions about the availability and structure of reference data, the strength of domain gaps, and the operational requirements of the downstream task:

• In UDC restoration, optimal PID assumes local rigidity, tractable domain adaptation, and the feasibility of dense correspondence; mismatches or occlusions may limit transfer (Feng et al., 2023).
• In medical imaging, effectiveness relies on the generator's ability to distinguish and erase pathology without corrupting normal anatomy, and on segmentor confidence—that is, synthetic data must be nearly indistinguishable from "true" healthy images (Du et al., 2021).
• Contrast encoding admits only $b+1$ analog levels, and as $b$ increases the distinguishability between gray levels diminishes; full digital recovery, while lossless, may require storing a "key" to map code blocks back to values (Ruderman, 2011).

A plausible implication is that PID solutions will increasingly blend purpose-built architecture (attention, domain/geometry alignment), semi-supervised learning, and human-aligned annotation (analog previews), providing a general recipe for data synthesis wherever direct labels are inaccessible, data are multi-domain, or long-term interpretability is paramount.