ADIEE: Vision & Medical Imaging Frameworks

• ADIEE is a dual framework system, featuring A-IDE for CT denoising with LLM-based expert routing and an image editing evaluation pipeline for automated dataset scoring.
• A-IDE leverages region-specialized RED-CNN experts and semantic routing to enhance denoising performance across variable anatomical regions, yielding superior PSNR, SSIM, and RMSE metrics.
• ADIEE’s image editing component automates dataset creation and quality scoring, enabling reliable model ranking, reward-driven fine-tuning, and scalable evaluation benchmarks.

ADIEE refers to two distinct advanced frameworks in contemporary computer vision and medical imaging research, each leveraging multimodal deep learning and/or agentic reasoning for their respective domains. The first, Agent-Integrated Denoising Experts (A-IDE), targets Low-Dose CT imaging; the second, Automatic Dataset Creation and Scorer for Instruction-Guided Image Editing Evaluation, addresses both dataset generation and automated quality scoring for instruction-driven image editing. Both frameworks are modular, data-centric, and algorithmically novel, with significant empirical gains in their applications (Cho et al., 21 Mar 2025, Chen et al., 9 Jul 2025).

1. Definitions and Scope

Agent-Integrated Denoising Experts (A-IDE) is a modular system for Low-Dose CT (LDCT) denoising, integrating multiple region-specialized convolutional denoisers with an LLM-based agent for scan routing. Its primary goal is robust generalization across anatomical regions with varying HU distributions and data scarcity (Cho et al., 21 Mar 2025).

ADIEE (Instruction-Guided Image Editing Evaluation) denotes a framework for automatic large-scale dataset construction and the supervised fine-tuning of a vision–language scorer for rating the success of text-driven image edits. Addressing the lack of public benchmarks, ADIEE enables reliable, automated, instruction-aware evaluation, outperforming both open-source and proprietary models (Chen et al., 9 Jul 2025).

2. System Architectures

2.1 A-IDE Framework

A-IDE comprises three RED-CNN (Residual Encoder–Decoder CNN) expert models and a GPT-4o-based LLM agent:

• Expert Models: Each RED-CNN is trained on a disjoint cluster of the Mayo-2016 CT dataset, partitioned according to anatomical region (abdomen, pelvis, lungs). Architecturally, these are 10-layer encoder–decoder CNNs (5 convolutional + 5 deconvolutional blocks, 5×5 kernels, residual skip-connections).
• Agentic Routing: The agent receives a semantic anatomical probability vector $\mathbf{p} \in \mathbb{R}^{20}$ (generated via BiomedCLIP cosine similarities between input image features and textual anatomical labels). Given region-specialized model descriptions, the agent uses a heuristic $$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$ to select the optimal expert.
• Training: Each expert minimizes an MSE loss plus weight decay:

$$\mathcal{L}_i(\theta_i) = \frac{1}{N_i} \sum_{n=1}^{N_i} \|f_{\theta_i}(x_n)-y_n\|_2^2 + \lambda\|\theta_i\|_2^2$$

with the total loss $$\mathcal{L}_{\mathrm{total}} = \sum_{i=0}^2 \mathcal{L}_i(\theta_i)$$.

2.2 ADIEE Framework

The ADIEE image editing evaluation system consists of:

• Dataset Generation Pipeline: Two branches—
  • Application of nine open-source instruction-driven editing methods to diverse datasets to generate synthetic candidate edits, scoring them via CLIP-based directional similarity (CLIP-D) and visual similarity thresholds (with CLIP-I, DINO-I).
  • Extraction of multi-turn edit sequences from SEED-Data-Edit, MagicBrush, etc., assigning fractional scores to intermediates via a piecewise function.
• Scorer Model: LLaVA-NeXT-8B is extended with a [SCORE] token and an MLP head for regression, producing a scalar value in [0,10] after a prompt concatenating both images and the instruction.
• Training Objective: Joint cross-entropy (text template) and $L_1$ regression loss, with weighting $\lambda_\text{txt}=1$, $\lambda_\text{score}=10$.

3. Datasets, Protocols, and Preprocessing

3.1 A-IDE

• Dataset: Mayo-2016 LDCT paired with full-dose CT, quarter-dose noise level, slice thickness 1 mm.
• Preprocessing: HU windowing $[-1000, 3000]$; patchification into $55 \times 55$ non-overlapping patches; PCA and $k$-means ($$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$0) on BiomedCLIP embeddings to form region clusters.
• Splits: For each expert/baseline, $$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$1 train, $$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$2 validation, $$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$3 test. Augmentation with in-plane rotations for data-limited clusters.

3.2 ADIEE

• Synthetic Method Data: $$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$4 samples from nine editing methods, using MagicBrush and Emu-Edit as primary sources.
• Multi-turn Data: $$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$5 samples from edit sequences, enabling score diversification.
• Total: Over $$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$6 (input, instruction, candidate edit, score) quadruplets.
• Labeling: Combinatorial application of CLIP-D and visual thresholds for synthetic, piecewise interpolation for multi-turn intermediates.

4. Evaluation Metrics and Results

4.1 A-IDE

• Metrics:
  • RMSE:

  $$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$7 - PSNR (dB):

  $$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$8 - SSIM:

  $$\hat{k} = \arg\max_k \sum_{j \in \mathcal{S}_k} p_j$$9

• A-IDE Results (mean ± std):

Model	RMSE	PSNR (dB)	SSIM
Baseline	0.097±0.00164	43.06±1.73	0.9557±0.0125
Expert 0	0.097±0.00245	42.15±2.28	0.9483±0.0200
Expert 1	0.086±0.00165	43.33±1.82	0.9576±0.0107
Expert 2	0.107±0.00196	40.89±1.39	0.9435±0.0147
A-IDE	0.094±0.00169	43.42±1.72	0.9583±0.0110

• Summary: A-IDE achieves the highest PSNR/SSIM and lowest RMSE, demonstrating superior generalization via expert routing.

4.2 ADIEE

• Benchmarks:

  • ImagenHub (pointwise): Spearman ρ computed on ratings from three annotators.
  • AURORA-Bench (pointwise and pairwise): Scalar ratings normalized to [0,1] and model/human preferences.
  • GenAI-Bench (pairwise): Model preference accuracy.
• Key Results:

Benchmark	Metric	Best Baseline	Proprietary	ADIEE
AURORA (ptwise)	ρ	0.2351 (Qwen2.5)	0.1052 (G1.5)	0.4734
GenAI (pwise)	Accuracy (%)	53.54 (GPT-4o)	55.93 (G1.5)	59.96
AURORA (pwise)	Accuracy (%)	50.81 (GPT-4o)	28.13 (G1.5)	55.56
ImagenHub	ρ	0.2351 (Qwen2.5)	0.2728 (G1.5)	0.3450

• Impact: ADIEE yields the highest correlations and pairwise preferences among open VLMs and outperforms Gemini-Pro 1.5 on all benchmarks.

5. Application Scenarios

5.1 A-IDE

• LDCT Denoising: Real-time, anatomy-specific enhancement in heterogeneous and low-data regimes.
• Overfitting Mitigation: Expert-wise specialization reduces representation collapse and improves robustness for minority anatomies.
• Automatic Routing: GPT-4o-based orchestration removes the need for manual protocol adjustment or a unified overparameterized model, accommodating anatomical label extensions with minimal engineering.

5.2 ADIEE

• Image Edit Scoring: Open-source, instruction-aware replacement for proprietary VLM judges, suitable for large-scale and cost-efficient evaluation pipelines.
• Model Selection and Leaderboard Ranking: Zero-shot best-edit selection and leaderboard generation matching human annotation orderings.
• Reward Model for RLHF: Used as a reward signal for diffusion model fine-tuning and prompt-conditioned training, yielding notable improvements in edit quality metrics (e.g., MagicBrush ImagenHub score increased by +8.98%).
• Automated Dataset Curation: Enables further research in edit evaluation and model alignment.

6. Limitations, Challenges, and Future Directions

• A-IDE:
  • Dependent on LLM routing correctness; misclassification may degrade performance.
  • Evaluated only at a single (quarter-dose) noise level; extending to varied radiation doses is an open area.
  • No adversarial or consistency regularization across experts. Future work includes multi-agent hyperparameter tuning (M3Builder) and end-to-end reinforcement learning for joint expert–agent optimization (Cho et al., 21 Mar 2025).
• ADIEE:
  • Current scoring is scalar and lacks decomposition into edit-specific sub-axes (e.g., semantic, artifact, or perceptual domains).
  • Reward signal is only applied at the final output step; future work may differentiate score gradients through entire pipelines.
  • Dataset diversity is bounded by the source editing methods and instruction complexity; extending to more diverse, in-the-wild editing scenarios could further improve robustness (Chen et al., 9 Jul 2025).

7. Significance and Outlook

Both A-IDE and ADIEE target core challenges in their respective fields by modularizing expertise and/or leveraging open-source multimodal models under agentic or data-centric protocols. A-IDE demonstrates that agent-specialized assembly of conventional architectures, routed via LLMs and semantic image–text embeddings, outperforms monolithic denoisers in multi-anatomy, data-scarce clinical tasks. ADIEE closes the public data gap in instruction-based image editing evaluation, introduces a reproducible and scalable scoring mechanism, and establishes new automated standards for model comparison, ranking, and reward-driven training. Both frameworks signal a paradigm shift toward agent-driven, data-scalable pipelines in vision and medical AI, with future progress tied to expanded datasets, model explainability, and joint optimization methodologies (Cho et al., 21 Mar 2025, Chen et al., 9 Jul 2025).