Prompt4Trust: Clinically Aligned Calibration in VQA

Updated 5 July 2026

Prompt4Trust is a framework that uses reinforcement learning and prompt augmentation to align verbalized confidence with actual accuracy in medical VQA.
It employs a dual-model system where a lightweight CGP generator crafts context-aware prompts for a frozen multimodal model, enhancing safety-critical decision support.
Empirical results on PMC-VQA show improved calibration and higher high-confidence accuracy, surpassing previous state-of-the-art performance.

Searching arXiv for the Prompt4Trust paper and closely related calibration work.

Searching arXiv for "Prompt4Trust clinically aligned confidence calibration multimodal LLMs medical VQA"

Prompt4Trust is a reinforcement learning prompt augmentation framework for clinically-aligned confidence calibration in multimodal LLMs (MLLMs), introduced for medical visual question answering (VQA) in settings where both answer correctness and the reliability of stated confidence are safety-critical. Its defining idea is to train a lightweight LLM to generate context-aware auxiliary prompts that steer a frozen downstream MLLM toward responses whose verbalized confidence more faithfully tracks empirical accuracy, with explicit priority placed on making high-confidence predictions genuinely dependable in clinical use (Kriz et al., 12 Jul 2025).

1. Problem setting and conceptual basis

Prompt4Trust is formulated for medical VQA instances of the form $(v, t, \mathcal{A}, y)$ , where $v$ is a 2D medical image, $t$ is a textual question, $\mathcal{A}=\{a_1,\dots,a_k\}$ is a multiple-choice answer set, and $y \in \mathcal{A}$ is the ground-truth answer (Kriz et al., 12 Jul 2025). The target model must output both an answer and a verbalized confidence score, and the paper treats calibration as the alignment between expressed confidence and accuracy for such verbalized confidence scores.

The framework is motivated by three linked deficiencies of contemporary MLLMs. First, these systems are highly sensitive to prompt wording and ordering, so small changes in instructions can alter both answers and stated confidence. Second, they are often overconfident when wrong, including in settings where hallucinated or unsupported answers may still be accompanied by very high numeric confidence. Third, generic calibration objectives do not capture the asymmetry of clinical risk: wrong answers with very high confidence are more dangerous than correct answers with conservative confidence. Prompt4Trust therefore adopts a clinically aligned notion of calibration in which very high confidence should be reserved for cases where accuracy is also very high, while low or uncertain confidence is preferable when evidence is ambiguous (Kriz et al., 12 Jul 2025).

This emphasis reflects the intended use of confidence by clinicians. The paper explicitly frames model confidence as a signal that may influence whether an answer is acted upon, escalated for review, or treated as uncertain. A plausible implication is that Prompt4Trust is less concerned with globally minimizing calibration error across all bins than with making the high-confidence region trustworthy enough for decision support.

2. Two-model architecture and prompt augmentation

Prompt4Trust separates calibration guidance from task execution through a two-model design. The downstream task model $f_\tau$ is a frozen multimodal model, such as Qwen2-VL-2B-Instruct, that consumes the image $v$ , the question and options $t \oplus \mathcal{A}$ , and an auxiliary prompt, and outputs an answer $\hat{y}$ together with a confidence $\hat{p}$ (Kriz et al., 12 Jul 2025). The auxiliary model $v$ 0 is a lightweight LLM, specifically Qwen2.5-1.5B-Instruct, that sees only the textual part $v$ 1 and generates a Calibration Guidance Prompt (CGP) $v$ 2.

The resulting data flow is: $v$ 3

$v$ 4

This division is central. The downstream MLLM is not fine-tuned; only the CGP generator is optimized. Prompt augmentation is therefore not a fixed manual template but a context-aware intervention conditioned on the specific question and answer choices. According to the paper, generated CGPs typically include instructions about reasoning procedure, careful consideration of alternatives, restraint in assigning very high confidence, and output formatting so that the answer and confidence can be parsed reliably (Kriz et al., 12 Jul 2025).

The framework defines confidence by asking the downstream MLLM to emit a score out of 100 and converting it to: $v$ 5 where $v$ 6. This turns verbalized confidence into a calibrated target variable without requiring access to internal logits or softmax probabilities.

3. Reinforcement learning objective and calibration metrics

Prompt4Trust is trained with reinforcement learning, using the frozen downstream MLLM as the environment and the CGP generator as the policy. The state is the text query $v$ 7 together with an instruction to generate a guidance prompt; the action is the token sequence constituting the CGP; and the reward is computed from the ground-truth label, the predicted answer, and the verbalized confidence (Kriz et al., 12 Jul 2025).

The reward function is: $v$ 8 with $v$ 9 and $t$ 0 (Kriz et al., 12 Jul 2025).

This reward makes the calibration objective explicitly asymmetric. Correct answers are rewarded more when confidence is high. Incorrect answers are penalized more severely as confidence increases, with an additional downward shift of $t$ 1. Invalid outputs are heavily penalized. In practical terms, the system is encouraged to be confident when right, cautious when wrong, and syntactically reliable in its output format.

The policy is optimized with Group-Relative Policy Optimization (GRPO), using 8 sampled completions per prompt, AdamW with learning rate $t$ 2, per-device batch size 8, and 4 optimization iterations per batch (Kriz et al., 12 Jul 2025). The CGP generator input is capped at 512 tokens, and the generated CGP at 256 tokens. Sampling uses temperature $t$ 3, top- $t$ 4, and $t$ 5 after grid search.

Evaluation uses standard metrics over valid predictions. Accuracy is

$t$ 6

while calibration is measured with Expected Calibration Error (ECE) and Brier score: $t$ 7

$t$ 8

ECE and Brier are computed only over valid predictions, whereas invalid outputs count as incorrect for accuracy (Kriz et al., 12 Jul 2025).

4. Empirical performance on PMC-VQA

Prompt4Trust is evaluated on PMC-VQA, a medical VQA benchmark containing 227k multiple-choice questions spanning diverse medical imaging modalities. The reported training setup uses 5,000 training samples, 1,000 validation samples, and 2,000 manually reviewed test samples (Kriz et al., 12 Jul 2025).

The main benchmark comparison with Qwen2-VL-2B-Instruct as the downstream model is summarized below.

Method	ECE / Brier	Accuracy (%)
Verbalized	0.517 / 0.523	40.40
Verbalized + Fixed-PA	0.493 / 0.496	44.25
Consistency (21 samples)	0.265 / 0.323	36.60
Avg-Conf (21 samples)	0.265 / 0.322	36.60
Prompt4Trust	0.163 / 0.264	45.95

These results show that Prompt4Trust improves both calibration and single-shot accuracy relative to naive verbalized confidence, fixed prompt augmentation, and sampling-based consistency baselines (Kriz et al., 12 Jul 2025). The paper further states that this accuracy surpasses the previous PMC-VQA leaderboard state of the art of 42.3% from MedVInT, while using 3.5B total parameters across the CGP generator and the downstream MLLM.

The calibration gains are especially pronounced in the clinically critical high-confidence regime. For predictions with $t$ 9, Prompt4Trust achieves 76.87% accuracy, compared with 44.74% for the verbalized baseline, 46.99% for Verbalized + Fixed-PA, 50.71% for Consistency, and 50.49% for Avg-Conf (Kriz et al., 12 Jul 2025). This makes the system’s most confident outputs markedly more trustworthy than those of the comparison methods.

The same pattern appears when examining confidence on incorrect answers. The verbalized baseline has mean confidence $\mathcal{A}=\{a_1,\dots,a_k\}$ 0 when wrong, and Verbalized + Fixed-PA still yields $\mathcal{A}=\{a_1,\dots,a_k\}$ 1. Prompt4Trust reduces this to $\mathcal{A}=\{a_1,\dots,a_k\}$ 2, indicating much lower and more stable confidence under error (Kriz et al., 12 Jul 2025). The paper interprets this as a tendency to admit uncertainty rather than doubling down on incorrect responses.

Calibration-curve analysis shows that Prompt4Trust stays close to the diagonal in the high-confidence region, whereas baseline methods are strongly overconfident near 1.0 (Kriz et al., 12 Jul 2025). At low confidence, Prompt4Trust is somewhat under-confident, with accuracy exceeding stated confidence. This suggests a conservative safety margin: low confidence becomes a strong cue that the answer should not be trusted without additional review.

A distinctive feature of Prompt4Trust is that the CGP generator is trained with a small downstream MLLM and then transferred zero-shot to larger models. The paper reports experiments replacing Qwen2-VL-2B-Instruct with Qwen2.5-VL-7B-Instruct and InternVL3-14B-Instruct while keeping the learned CGP generator fixed (Kriz et al., 12 Jul 2025). For Qwen2.5-VL-7B-Instruct, Prompt4Trust reaches ECE 0.289, Brier 0.317, accuracy 48.65%, and 74.65% accuracy on predictions with confidence at least 0.85. For InternVL3-14B-Instruct, it reaches ECE 0.302, Brier 0.327, accuracy 51.17%, and 63.97% high-confidence accuracy (Kriz et al., 12 Jul 2025). The main effect is not uniformly higher overall accuracy, but better-calibrated confidence in the region where confidence matters most.

Within calibration research, Prompt4Trust is positioned against several alternatives. Conventional calibration techniques generally operate on logits or softmax outputs and target global calibration error, whereas Prompt4Trust treats the MLLM as a black box and optimizes verbalized confidence under a clinically asymmetric reward (Kriz et al., 12 Jul 2025). Manual prompt-based calibration approaches, including verbal self-assessment and fixed “do not be over-confident” prompts, remain brittle and prompt-sensitive. Sampling-based approaches such as consistency estimation can improve calibration, but in the reported experiments they require 21 samples per question and still underperform Prompt4Trust on both single-shot accuracy and high-confidence reliability (Kriz et al., 12 Jul 2025). Relative to broader RL prompt optimization work, the novelty lies in explicitly optimizing for clinically aligned confidence calibration rather than generic task performance.

Prompt4Trust also sits within a wider research trend toward explicit trust signals in AI systems. WebTrust assigns a continuous reliability score in $\mathcal{A}=\{a_1,\dots,a_k\}$ 3 to web statements and pairs it with textual justification, framing trust as a scalar estimate of factual reliability rather than an implicit judgment (Chandra et al., 5 Jun 2025). A user study on RAG explanations reports that source attribution, factual grounding, and information coverage guide users toward selecting higher-quality responses, while also showing that trust is shaped by clarity, actionability, and prior knowledge rather than objective quality alone (Łajewska et al., 20 Jan 2026). AraTrust operationalizes trustworthiness for Arabic LLMs as multi-dimensional accuracy across truthfulness, ethics, safety, unfairness, privacy, and offensive language, illustrating how trust benchmarks can be language- and culture-specific (Alghamdi et al., 2024). “Compliance as a Trust Metric” formalizes policy obligations over event logs and derives a dynamic compliance score in $\mathcal{A}=\{a_1,\dots,a_k\}$ 4, treating trust as auditable behavior rather than subjective opinion (Wu et al., 3 Jan 2026). These parallel developments suggest that Prompt4Trust is part of a broader shift from opaque confidence displays toward explicit, measurable, and domain-aligned trust instrumentation.

6. Limitations, safety implications, and outlook

Prompt4Trust does not alter the downstream MLLM’s underlying factual knowledge; it changes prompting and the expression of confidence rather than the model’s parametric beliefs (Kriz et al., 12 Jul 2025). The downstream model can still hallucinate, and the framework does not provide perfect calibration. Residual overconfidence remains possible, and the empirical evidence is confined to PMC-VQA rather than real clinical deployment.

The framework is also domain-specific in its reward design. Its asymmetry is tailored to clinical risk, where overconfident errors are especially costly. This makes the approach compelling for safety-critical decision support, but it also implies that transfer to other domains would likely require re-specifying the reward to reflect different harm profiles. A plausible implication is that Prompt4Trust is best understood not as a universal calibration recipe but as a template for domain-aligned prompt optimization.

Even with those limitations, the system demonstrates an important design principle. Confidence calibration can be optimized through automated prompt engineering alone, without fine-tuning the downstream MLLM, and the resulting prompts can generalize from a smaller model to larger ones (Kriz et al., 12 Jul 2025). In that sense, Prompt4Trust provides both a concrete medical VQA method and a more general paradigm: use reinforcement learning over auxiliary prompts to align verbalized confidence with the trust expectations of a high-stakes application domain.