MetaSR: Content-Adaptive Metadata Orchestration for Generative Super-Resolution

Abstract: We study generative super-resolution (SR) in real-world scenarios where content and degradations vary across domains, genres, and segments. For example, images and videos may alternate between text overlays, fast motion, smooth cartoons, and low-light faces, each benefiting from different forms of side information. Existing metadata-guided SR methods typically use a fixed conditioning design, which is suboptimal when useful cues are content dependent and transmission budgets are limited. We propose MetaSR, a Diffusion Transformer (DiT)-based framework that selects and injects task-relevant metadata to guide SR under resource constraints. Specifically, we use the DiT's own VAE and transformer backbone to fuse heterogeneous metadata, and adopt an efficient distillation strategy that enables one-step diffusion inference. Experiments across diverse content buckets and degradation regimes show that MetaSR outperforms reference solutions by up to 1.0~dB PSNR while achieving up to 50\% transmission bitrate saving at matched quality. We assess these gains under a rate--distortion optimization (RDO) framework that jointly accounts for sender-side bitrate and receiver/display quality metrics (e.g., PSNR and SSIM).

• The paper introduces a sender–receiver framework where the sender compresses and transmits metadata (e.g., edges, depth) to complement low-resolution inputs for SR.
• It leverages a Diffusion Transformer with unified metadata token fusion to reduce posterior uncertainty and enhance reconstruction quality.
• Experimental results demonstrate up to 1.0 dB PSNR improvement and 50% bitrate savings under severe channel degradation.

Content-Adaptive Metadata Orchestration for Generative Super-Resolution: MetaSR

Sender–Receiver Framework for Generative SR

MetaSR formalizes super-resolution (SR) as a sender–receiver collaborative problem, departing from the tradition of treating SR as a purely receiver-side, pixel-based task. In realistic networked deployments, high-fidelity SR is limited by incomplete observations and bandwidth constraints. The sender performs content analysis and generates/compresses compact metadata (e.g., edges, depth) as an auxiliary stream alongside low-resolution (LR) content. The receiver then fuses decoded metadata with LR input for generative SR, utilizing resource-adaptive conditioning to minimize posterior uncertainty under rate–distortion objectives.

Figure 1: Sender–receiver pipeline—sender generates/compresses metadata, receiver adaptively fuses transmitted metadata with LR content for generative SR.

This system-level perspective is aligned with information-theoretic reconstruction, which acknowledges that irrecoverable high-frequency details cannot be deterministically inferred from LR pixels alone. Structured metadata thus functions as actionable side information, enabling more robust reconstruction, especially under severe transmission degradations.

Unified Metadata Conditioning via Diffusion Transformer (DiT)

MetaSR leverages the DiT architecture (Diffusion Transformer), specifically CogVideoX-2B, as its generative backbone. Metadata modalities (e.g., Canny edges, depth maps) are encoded via a shared VAE into latent tokens, which are then fused—alongside LR and text tokens—through the native DiT blocks with spatially consistent positional encodings. Crucially, heterogeneous metadata is integrated without architectural modification, and multi-modal attention enables flexible spatial/semantic fusion.

Figure 2: MetaSR’s unified metadata projection through DiT modules; qualitative examples show improved SR under Canny/depth guidance.

The pipeline employs a two-stage training strategy: latent-space adaptation followed by pixel-space refinement. The design accommodates variation in metadata types and transmission conditions without network redesign, resulting in a model-agnostic conditioning interface suitable for practical sender–receiver deployments.

Information-Theoretic Analysis: Entropy Suppression

MetaSR’s core claim is grounded in information theory: informative metadata reduces posterior entropy in HR reconstruction. Given a target image $X$, degraded input $Y$, and metadata $M$, the conditional entropy satisfies $H(X|Y,M) \leq H(X|Y)$. The reduction equals the conditional mutual information $I(X;M|Y)$, which is bounded above by the metadata's entropy. This yields explicit guidance: bitrate allocation for metadata should maximize $I(X;M|Y)$ under transmission constraints.

Verification-gated conditioning is introduced to handle unreliable metadata. If metadata fails a reliability gate, the model defaults to pixel-only conditioning, ensuring that entropy never increases and log-likelihood/optimal training loss is always minimized relative to the no-metadata baseline.

Posterior Uncertainty: Compression–Transmission–Generation Pipeline

MetaSR demonstrates that generative SR models are subject to unavoidable posterior uncertainty under practical compression and channel degradations. High-frequency information may be lost to compression artifacts or noise, and diffusion-based generative models (e.g., StableSR, DOVE) can introduce prior-driven hallucination, deviating from true HR.

Figure 3: Quality degradation in the compression–transmission–generation pipeline, highlighting prior-driven hallucination effects across SR baselines.

The sender–receiver design, with explicit metadata transmission, is shown to effectively suppress such ambiguity, measurable across metrics like PSNR, SSIM, LPIPS, and DISTS.

Rate–Distortion Evaluation and Numerical Results

MetaSR is evaluated under controlled scenarios where the sender transmits a JPEG-compressed base layer supplemented with compressed metadata streams (Canny edges via JBIG2). Experiments span noise-free (NN), low-noise (LN), and high-noise (HN) transmission regimes, quantifying the impact of metadata on the rate–distortion curves.

Figure 4: RDO curves across NN/LN/HN regimes—MetaSR achieves up to 1.0 dB PSNR gain and 50% bitrate saving over DOVE, especially under severe degradation.

Results indicate that the performance gap in favor of MetaSR increases as channel corruption rises; in HN conditions, MetaSR attains up to 1.0 dB PSNR improvement and achieves substantial bitrate savings for matched quality. Gains are most visible in structure-aware and perceptual metrics, confirming the effective suppression of posterior uncertainty.

Extending Metadata Conditioning Beyond SR

MetaSR’s metadata orchestration paradigm is extendable to tasks beyond SR. Preliminary results demonstrate edge-guided video frame interpolation, indicating the potential for broader impact in restoration tasks, provided efficient sender-side compression and adaptive reliability-aware gating.

Figure 5: Metadata-guided frame interpolation—demonstrates applicability of MetaSR-style conditioning beyond SR.

Implications and Future Directions

MetaSR’s content-adaptive, bitrate-constrained metadata orchestration addresses a fundamental gap in generative SR for networked deployments. The unified DiT-native token fusion interface allows flexible integration of heterogeneous metadata streams, providing robust gains under sender–receiver constraints, and is compatible with rapid one-step inference for practical receiver-side processing.

From a theoretical perspective, MetaSR formalizes the value of metadata in SR as conditional mutual information and aligns with classic source coding with side information (e.g., Wyner–Ziv). Practically, the approach enables efficient allocation of bitrate, reducing required transmission cost for target fidelity, and mitigates prior-driven hallucination.

For future research, challenges remain in video-native metadata compression, temporally consistent orchestration, and broader metadata families. Extending reliability-aware gating and adaptive bitrate allocation will further improve robustness. The sender–receiver framework is generalizable to other generative AI restoration tasks, promising improved operational efficiency and quality for real-world networked systems.

Conclusion

MetaSR substantiates content-adaptive metadata orchestration as an effective strategy for generative super-resolution in sender–receiver deployments. The framework delivers a unified and model-agnostic conditioning mechanism, validated both theoretically and empirically. Substantial improvements in structure-aware and perceptual quality, as well as practical bitrate savings under challenging transmission conditions, confirm the efficacy of structured metadata in reducing posterior uncertainty and ambiguity. Ongoing developments aim to expand the scope and efficiency of metadata-guided generative restoration across diverse tasks and channels.