MetaLLMiX: Zero-Shot HPO for Med Imaging

• MetaLLMiX is a zero-shot hyperparameter and model selection framework that uses meta-learning, SHAP explainability, and local LLM reasoning to bypass iterative searches.
• It employs an XGBoost meta-learner, TreeSHAP for feature attributions, and FAISS to retrieve historical experiments, enabling precise configuration recommendations.
• Empirical results on 8 medical imaging datasets show competitive accuracy with significantly reduced recommendation latency and training times compared to iterative methods.

Searching arXiv for the primary paper and a few related LLM-HPO references mentioned in the provided material. arXiv search query: "MetaLLMiX hyperparameters optimization medical imaging" MetaLLMiX is a zero-shot hyperparameter and model selection framework for deep learning that combines meta-learning, SHAP-based explainable AI, and efficient local LLM reasoning to recommend pretrained models and training hyperparameters without additional search trials. It is formulated for medical imaging transfer learning, where historical experiment outcomes are used as a meta-dataset, an XGBoost meta-learner predicts expected performance, TreeSHAP provides local and global feature attributions, FAISS retrieves similar prior experiments, and a lightweight open-source LLM produces a single configuration in strict JSON together with an explanation. The paper uses both the spellings MetaLLMix and MetaLLMiX; MetaLLMiX is the intended name in the abstract, and both denote the same system (Tiouti et al., 11 Sep 2025).

1. Problem setting and conceptual position

Hyperparameter optimization in deep learning is presented as difficult because the search space is large, the objective landscape is task-dependent and non-convex, and each candidate evaluation requires an expensive training run. The paper emphasizes that these difficulties are amplified in medical imaging by dataset heterogeneity, including modality, class imbalance, and resolution, together with a requirement for transparent decision-making (Tiouti et al., 11 Sep 2025).

MetaLLMiX is positioned against two existing families of methods. Conventional AutoML systems reduce manual effort, but still depend on many evaluations and provide limited task-level interpretability. Prior LLM-based HPO methods reduce random exploration through in-context reasoning, but still rely on iterative proposal-evaluate cycles and often on proprietary APIs, with associated token cost, latency, and limited output control. MetaLLMiX defines its central intervention as zero-shot HPO: no additional search trials are run on the target dataset; instead, a single configuration is recommended in one shot and then trained once.

A common point of confusion is nomenclature. MetaLLMiX is distinct from "MetaMix: Improved Meta-Learning with Interpolation-based Consistency Regularization" (Chen et al., 2020), which concerns mixup-based regularization for MAML-style few-shot learning rather than LLM-guided hyperparameter selection. The 2025 MetaLLMiX framework is specifically a meta-learning and XAI aided system for model and hyperparameter recommendation (Tiouti et al., 11 Sep 2025).

2. Architecture, data representation, and decision pipeline

The framework begins with construction of a historical experiment database. Each entry records dataset meta-features, the chosen pretrained CNN base model, training hyperparameters, and resulting metrics such as train, validation, and test accuracy, precision, recall, F1, and total training time (Tiouti et al., 11 Sep 2025). The paper describes the dataset descriptors as dataset meta-features (7), while enumerating total_images, num_classes, class_imbalance_ratio, class_entropy, mean_class_size, std_class_size, min_class_size, max_class_size, and modality (CT/[MRI](https://www.emergentmind.com/topics/metacognitive-readiness-index-mri)/X-ray/US/DMS/[OCT](https://www.emergentmind.com/topics/neutrino-octupole-oct)). That enumeration is the operative feature set used throughout the method.

The searchable configuration space includes seven ImageNet-pretrained CNN backbones: ResNet50, Xception, EfficientNetB5, EfficientNetB0, DenseNet121, InceptionV3, and NASNetMobile. Fine-tuning strategy is chosen from feature extraction, partial (last 10/30 layers), or full. The classifier head uses dense_units ∈ {512, 1024, 1536}, dropout ∈ {0.3, 0.4, 0.5}, optimizer ∈ {Adam, SGD, RMSprop}, learning_rate ∈ {1e−3, 1e−4, 1e−5}, and batch_size ∈ {16, 32, 64}. For the transfer-learning classifier, the head is GAP → Dense(dense_units) → BatchNorm + [ReLU](https://www.emergentmind.com/topics/rectified-linear-unit-relu-regression) + Dropout → Dense-softmax.

The decision pipeline is modular. XGBoost is trained as a meta-learner to predict expected performance from the concatenated dataset meta-features and candidate configuration. TreeSHAP is then used to compute per-feature attributions. FAISS retrieves the top-8 most similar historical experiments on normalized meta-feature vectors. A local open-source LLM, run via Ollama, receives a structured prompt containing the target dataset meta-features, the retrieved experiments, and a SHAP summary, and must return a strict JSON configuration followed by a plain-text explanation. A separate evaluator, described as an LLM-as-judge, scores the output on format adherence, accuracy, completeness, conciseness, and fluency on a 0–4 scale, with three repeated evaluations averaged to reduce variance (Tiouti et al., 11 Sep 2025).

Component	Function	Implementation
Meta-learner	Predict expected performance from (m, \theta)	XGBoost
XAI module	Compute local/global attributions and directional effects	TreeSHAP
Retrieval	Supply similar prior experiments as evidence	FAISS top-8
Generator	Produce one recommended configuration and explanation	Local LLM via Ollama
Output control	Score adherence, accuracy, completeness, conciseness, fluency	LLM-as-judge

This architecture is designed to remove the trial-and-error loop characteristic of standard HPO. A plausible implication is that the recommendation stage becomes primarily an inference and retrieval problem over accumulated experimental knowledge rather than an online optimization process.

3. Formalization and optimization logic

The paper states the validation objective for HPO as

$$J(\theta) = \mathbb{E}_{(x,y) \sim D_{\text{valid}}} \left[ \mathcal{L}\big(f(x;\theta), y\big) \right].$$

Traditional HPO minimizes $J(\theta)$ through repeated training and evaluation of many candidate $\theta$. MetaLLMiX instead approximates an optimal configuration in one step through meta-knowledge and XAI:

$\theta^* \approx g_{\phi}\big(m(D)\big),$

where $m(D)$ denotes the dataset meta-features and $g_\phi$ is the recommendation mapping induced by the learned components (Tiouti et al., 11 Sep 2025).

At the meta-learning level, the objective is described as training the meta-learner to predict validation loss or accuracy from $(m,\theta)$, operationally realized by fitting XGBoost and then using SHAP to determine which components of $\theta$ and which regions of meta-feature space are beneficial. The paper also provides the Shapley-value formulation

$$\phi_{j} = \sum_{S \subseteq F \setminus \{j\}} \frac{|S|!\,(|F|-|S|-1)!}{|F|!} \Big( f_{S \cup \{j\}}\big(x_{S \cup \{j\}}\big) - f_{S}\big(x_{S}\big) \Big),$$

with TreeSHAP used in practice for fast, exact attributions in tree ensembles.

The end-to-end algorithm is concise and sequential. Meta-features are computed for the target dataset, the historical database is loaded, global SHAP summaries are aggregated, FAISS retrieves the nearest experiments, and the prompt is assembled. The local LLM then generates a JSON configuration and explanation; the judge evaluates the response; the recommended configuration is parsed and trained exactly once; and the resulting outcome is appended back into the database together with a local SHAP explanation. This update step makes the framework incrementally extensible: each zero-shot deployment can become new meta-training evidence for subsequent recommendations (Tiouti et al., 11 Sep 2025).

4. Experimental design and evaluation protocol

The empirical study covers eight medical imaging classification datasets: Alzheimer (MRI, 6,400 images, 4 classes), Brain Tumor (MRI, 3,264 images, 4 classes), Breast Cancer (Ultrasound, 780 images, 3 classes), Chest CT (CT, 1,000 images, 4 classes), Chest COVID X-Rays (X-ray, 5,910 images, 2 classes), Eye Disease OCT (OCT, 4,217 images, 4 classes), Kidney Diseases (CT, 12,446 images, 4 classes), and Skin—ISIC (Dermoscopy, 2,357 images, 9 classes) (Tiouti et al., 11 Sep 2025).

Nine locally deployed open-source LLMs are evaluated through Ollama: deepseek-r1:7b, llama3.1:8b, gemma3:4b, codellama:7b, qwen2.5-coder:7b, qwen3:8b, codegemma:7b, mistral:7b, and deepseek-coder:6.7b. Baselines are Random Search and Bayesian Optimization with a Gaussian-process surrogate via scikit-optimize, each run for 20 iterations. Other frameworks such as Hyperband, BOHB, Optuna, and Ray Tune are explicitly noted as not having been evaluated in this study.

Evaluation uses four classes of measurements. First, predictive utility is reported via 4-fold cross-validation test accuracy. Second, optimization response time measures seconds required to produce a recommendation. Third, training time measures the cost of fitting the final model under the recommended configuration. Fourth, the judge model assigns 0–4 scores for completeness, fluency, conciseness, consistency, and format, averaged over three runs. The paper explicitly states that AUC and token or cost measurements were not reported (Tiouti et al., 11 Sep 2025).

Dataset	Modality	Images / Classes
Alzheimer	MRI	6,400 / 4
Brain Tumor	MRI	3,264 / 4
Breast Cancer	Ultrasound	780 / 3
Chest CT	CT	1,000 / 4
Chest COVID X-Rays	X-ray	5,910 / 2
Eye Disease OCT	OCT	4,217 / 4
Kidney Diseases	CT	12,446 / 4
Skin—ISIC	Dermoscopy	2,357 / 9

The protocol is therefore not a comparison of search budget efficiency under identical iterative loops; rather, it is a comparison between a single-shot recommendation procedure and two 20-iteration search procedures, using accuracy, recommendation latency, and final training time as the principal axes of evaluation.

5. Quantitative performance and model behavior

MetaLLMiX is reported to achieve optimal or tied-best accuracy on 5 of 8 tasks—Brain, Breast, Covid-Xrays, Kidney, and Skin-ISIC—and to remain within 1–5% of the best baseline on the remaining tasks. The paper gives representative gaps such as Alzheimer 0.92 vs 0.93, Chest CT 0.92 vs 0.96, and Eye Disease 0.91 vs 0.92 (Tiouti et al., 11 Sep 2025).

Recommendation latency is a central result. The framework returns recommendations in 7.55–13.51 seconds, corresponding to a 99.6–99.9% reduction relative to Random Search (2,033–17,085 s) and Bayesian Optimization (2,637–18,145 s). Final-model training time is also favorable: MetaLLMiX is fastest on 6 of 8 datasets, with 2.4× to 15.7× lower training times than traditional baselines. The paper gives concrete examples: Alzheimer 88.19 s versus Random Search 549.59 s and Bayesian Optimization 140.45 s; Covid-Xrays 122.69 s versus Random Search 370.78 s and Bayesian Optimization 1,257.37 s; Eye Disease 122.06 s versus Random Search 508.86 s and Bayesian Optimization 1,030.84 s.

Performance varies with the local LLM. gemma3:4b often produced the fastest response times, qwen2.5-coder:7b frequently ranked highest overall across accuracy, output format, and explanation quality, and qwen3:8b was slower. The paper treats this as evidence that local-LLM choice is itself a deployment variable rather than an interchangeable implementation detail.

The interpretability mechanism is illustrated on the Alzheimer dataset. The SHAP summary reported positive architecture effects for EfficientNetB0 (+0.026) and ResNet50 (+0.022), and negative effects for NASNetMobile (−0.144) and Xception (−0.032). An example recommendation for Alzheimer MRI with four classes and moderate imbalance is ResNet50 or EfficientNetB0, learning rate 1e−4, batch size 32, dropout 0.4, dense units 1024, optimizer SGD, and partial fine-tuning of the last_10 layers. The stated rationale is that SHAP favors ResNet50 and EfficientNetB0, while retrieved neighbors with similar meta-features support these settings and indicate reduced training time (Tiouti et al., 11 Sep 2025).

6. Interpretability, limitations, and future directions

Interpretability is not an auxiliary add-on in MetaLLMiX; it is one of the coordinating signals in the recommendation loop. TreeSHAP provides both local and global feature attributions, the system aggregates SHAP magnitudes by value and computes Pearson correlations to summarize directional effects, and the LLM is expected to ground its explanation in these summaries and in retrieved historical evidence. The LLM-as-judge then penalizes shallow or malformed outputs. This makes the framework not only a recommender of $\theta$ but also a controller of explanation structure and evidence use (Tiouti et al., 11 Sep 2025).

The main limitations arise from data coverage and distribution shift. Historical experiments are randomly sampled rather than optimally designed, so the quality of the meta-learner and SHAP summaries depends directly on the diversity and density of the meta-dataset. Recommendations may degrade when a new dataset lies outside the distribution of stored meta-features. Hard multi-class tasks and highly imbalanced datasets can expose LLM variability, and the paper notes that some LLMs may misinterpret context or overfit to misleading patterns in sparse regions. The feature set is also acknowledged as handcrafted and potentially incomplete: the listed dataset-level descriptors may not capture texture, pathology-specific cues, or other task-relevant structure.

A further limitation is methodological rather than numerical. The study does not report formal ablations removing SHAP or RAG, so the isolated contribution of each subsystem is not quantified. The judge model is itself another LLM and is therefore subject to bias and variance, even though repeated evaluations and emphasis on format and accuracy are intended to reduce hallucinations. These caveats imply that MetaLLMiX should be understood as a composite decision pipeline whose effectiveness depends on the interaction among retrieval quality, meta-dataset coverage, SHAP fidelity, prompt design, and the chosen local LLM.

The paper’s future directions are correspondingly broad: extension beyond medical imaging to computer vision, tabular data, NLP, and time series through larger multi-domain meta-datasets; few-shot or online adaptation through continual logging of new single-shot outcomes; multi-objective HPO incorporating accuracy, memory, cost, and training time; and specialized or mixture-of-experts LLMs to reduce variability and improve generalization. Within the scope reported, MetaLLMiX is best characterized as an attempt to transform HPO from repeated black-box optimization into a one-shot, retrieval- and explanation-guided meta-inference procedure (Tiouti et al., 11 Sep 2025).