4KAgent: Agentic Super-Resolution System

• 4KAgent is an agentic super-resolution system that restores low-res images to 4K with iterative refinement and quality-driven expert selection.
• Its modular architecture integrates a Profile, Perception, and Restoration Agent to dynamically plan and execute image enhancement tasks.
• Extensive benchmarking across 11 tasks and 26 benchmarks confirms its state-of-the-art performance in fidelity and perceptual quality.

4KAgent is a unified agentic super-resolution generalist system developed to universally upscale any image—including highly degraded, low-resolution, and domain-unexpected inputs—to 4K resolution and beyond, using iterative refinement. It employs a modular architecture featuring agentic planning, execution-reflection cycles, a specialized face restoration pipeline, and a quality-driven mixture-of-experts policy. Through rigorous benchmarking across 11 task categories and 26 benchmarks—including natural images, AI-generated content, medical imaging, and remote sensing—4KAgent achieves state-of-the-art performance on both fidelity and perceptual metrics, establishing a new agentic paradigm for low-level vision tasks (2507.07105).

1. System Structure and Components

4KAgent is instantiated as a multi-agent super-resolution (SR) framework, composed primarily of three interacting modules:

• Profile Module: Functions as a “system prompt,” allowing users to specify restoration parameters, such as the chosen Perception Agent LLM/VLM (e.g., DepictQA, Llama-3.2-Vision), scale factors (2×, 4×, 8×, 16×), restoration options (predefined or dynamic), face restoration activation, image brightening, and the trade-off between perceptual quality and fidelity.
• Perception Agent: Operates as a four-stage analyzer, bridging low-level image quality assessment (IQA) and high-level, vision-language reasoning. It:
  • Extracts IQA metrics (CLIPIQA, TOPIQ, MUSIQ, NIQE),
  • Applies a VLM/LLM to identify degradations and generate a restoration agenda and sequential plan,
  • Produces a restoration plan (“P_I”) specifying the sequence of domain-specific restoration operations (e.g., denoising, deblurring, deraining, artifact removal, super-resolution).
• Restoration Agent: Executes the restoration agenda through stepwise application of multiple expert restoration tools. At each step, the agent:
  • Applies several restoration models (e.g., denoiser, deblurrer, upscaler),
  • Utilizes a quality-driven mixture-of-experts (Q-MoE) policy, selecting the candidate output with the highest joint quality score,
  • Incorporates an execution-reflection cycle: outputs are evaluated, and rollback is triggered if the quality falls below a threshold,
  • Embeds a dedicated Face Restoration Pipeline, segmenting and cropping detected faces before applying specialized tools, and reseating the optimal facial restoration considering clarity and identity preservation.

This compositional architecture is depicted in the system overview; the Profile module controls configuration, the Perception Agent formulates the restoration plan, and the Restoration Agent conducts and evaluates iterative restoration.

2. Agentic Execution and Quality-Driven Mixture-of-Experts

The agentic character of 4KAgent derives from its explicit reasoning and dynamic planning, driven by LLM/VLM modules. Restoration proceeds under a recursive execution-reflection-rollback cycle:

• For each restoration step $k$, starting from image $I_{k-1}$, a set of $N$ tools $\{T_1,\dots,T_N\}$ output restoration candidates $R_i = T_i(I_{k-1})$.
• Each $R_i$ is evaluated with a composite quality score $Q_s$ combining HPSv2 and no-reference IQA metrics (NIQE, MUSIQ, MANIQA, CLIPIQA).
• The new image $I_k$ is selected by maximizing quality:

$I_k = \operatorname{argmax}_i Q_s(R_i)$

• If $Q_s(I_k)$ falls below a threshold $\eta$, rollback is enacted and the restoration plan is dynamically updated by the agent.

Face restoration occurs after general SR steps when enabled. Cropped faces are processed by a face toolbox, and selections are made by quality functions $Q_s^f$ incorporating both perceptual indices (e.g., CLIB-FIQA) and identity preservation (measured by cosine similarity in ArcFace feature space). The restored faces are reintegrated into the upscaled image.

The following pseudocode summarizes the pipeline:

Q_I = compute_IQA_metrics(input_image) D_I, A_I = VLM_reasoning(Q_I, input_image) P_I = generate_restoration_plan(D_I, A_I, prior_experience) for k, task in enumerate(P_I): candidates = [T_i(image_k_minus_1) for T_i in toolbox] quality_scores = [Q_s(candidate) for candidate in candidates] image_k = candidates[argmax(quality_scores)] if Q_s(image_k) < eta: rollback() update_plan() faces = detect_faces(image_k) for face in faces: face_candidates = [face_tool(face) for face_tool in face_toolbox] face_scores = [Q_s_f(candidate) for candidate in face_candidates] best_face = face_candidates[argmax(face_scores)] image_k = paste_face(image_k, best_face)

3. Evaluation Metrics and Tuning

4KAgent’s evaluation uses both fidelity and perceptual metrics, allowing for user-driven profiling:

• Fidelity Metrics: PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index Measure). These measure signal fidelity and structure preservation.
• Perceptual Metrics: LPIPS, DISTS, and no-reference IQA metrics such as NIQE, MUSIQ, CLIPIQA, MANIQA, and HPSv2. These assess subjective and feature-level similarity as well as “realness” without ground-truth.
• Parameterizing the system profile (e.g., favoring fidelity vs. perceptual quality) directly modulates these metrics, supporting application-driven performance tuning.

4. Benchmarking and Results Across Task Categories

Extensive experiments spanning 11 tasks and 26 diverse benchmarks validate 4KAgent's generality and performance:

• Classical Super-Resolution: Tested on Set5, Set14, B100, Urban100, Manga109. In fidelity mode, 4KAgent ranks among top performers for PSNR and SSIM; in perception mode, it outperforms on perceptual metrics compared to SwinIR, HAT-L, X-Restormer, AgenticIR.
• Real-World SR: On RealSR, DrealSR, configurations such as ExpSR-s4-P and GenSR-s4-P yield state-of-the-art perceptual scores and sharper outputs relative to StableSR, SinSR.
• Multiple-Degradation Restoration: On MiO100 datasets (Groups A-C), the agentic design and Q-MoE+rollback consistently deliver state-of-the-art PSNR and IQA, outperforming AirNet, PromptIR, MiOIR, AgenticIR, MAIR.
• Face Restoration: Using WebPhoto-Test, enabling face restoration (profiles ExpSRFR-s4-P, GenSRFR-s4-P) yields improvements in face-specific IQA (CLIB-FIQA, DSL-FIQA) and general metrics.
• High-Factor SR: On challenging 16× (e.g., 256×256→4096×4096 on DIV4K-50), surpasses DiffBIR, OSEDiff on MUSIQ, NIQE, CLIPIQA, maintaining detail and realism.
• AIGC 4K SR: Applied to 1K AI-generated art (SANA, Stable Diffusion 3, PixArt-Σ, GPT-4o, FLUX.1-dev), 4KAgent's 4K outputs show improved fidelity and perceptual realism, as measured by MUSIQ-P and qualitative comparisons.
• Scientific Imaging: Excels at remote sensing (AID, DIOR, DOTA, WorldStrat), fluorescence microscopy (SR-CACO-2), pathology (bcSR), and medical imaging (X-ray, ultrasound, fundoscopy)—matching or surpassing state-of-the-art in SSIM, FSIM, NIQE, and preserving diagnostically relevant details.

Ablation studies confirm the contribution of Q-MoE and face-pipeline design choices to overall system performance.

5. Practical Applications

Universal and flexible, 4KAgent facilitates a broad range of applications:

• Media Streaming: Supports efficient transmission of low-resolution content with device-side upscale to 4K, reducing bandwidth and storage demands.
• Video Conferencing: Enhances clarity for low-res webcam/video feeds.
• Surveillance/Security: Improves facial and license plate detail in compressed, low-res (e.g., dashcam, CCTV) footage.
• Gaming/Entertainment: Enables real-time SR (analogous to DLSS) with GPU acceleration.
• XR/VR/AR: Delivers high-quality or enhanced passthrough imagery, compensating for modest capture resolutions.
• AIGC & Digital Art: Upscales and augments AI-generated image and video outputs.
• Scientific Imaging: Benefits remote sensing, fluorescence microscopy, pathology, and medical imaging (e.g., higher clarity in X-ray, ultrasound, fundus).

6. Distinctive Innovations

4KAgent introduces several novel elements:

• Agentic Planning: It is the first agentic generalist SR framework that dynamically analyzes degradations and plans restoration with no retraining per domain.
• Q-MoE Selector: Leverages a diverse toolset of restoration experts, using quality-driven selection for execution and reflection.
• Flexible Profiling: The Profile module provides rapid tuning between fidelity, perception, and specialized pathways (e.g., face restoration), all at inference time.
• Extensible Toolbox: Modular design accommodates expansion to new domains or expert models without retraining.
• State-of-the-Art Generalization: Demonstrates universal applicability and strong empirical results across classical, real-world, AIGC, and scientific/medical imaging benchmarks.

7. Future Research Directions

The framework prioritizes several areas for further development:

• Efficiency Optimizations: Strategies to parallelize independent operations and refine perception models are under development to reduce computational cost and facilitate real-time deployment.
• Safety, Fairness, Robustness: Addressing privacy and accuracy concerns in sensitive domains (e.g., surveillance, medicine) forms a research priority.
• Toolbox Expansion: Incorporating new domain-specific models (e.g., for video SR or novel scientific tasks) and text-driven restoration methods is planned.
• Dynamic Agentic Schemes: Continued research into more responsive execution-reflection-rollback protocols and advanced quality assessment will enhance adaptability and performance.
• Broad Deployment: Adaptations for mobile, edge, and real-time contexts (e.g., robotics, autonomous vehicles) are anticipated with improved efficiency and robustness.

4KAgent embodies a systematic integration of agentic AI concepts, quality-driven optimization, and domain-agnostic applicability, advancing the state of agent-based low-level vision and universal super-resolution methodologies (2507.07105).