Procedural Quantity Reward in CardiacMind

• Procedural Quantity Reward (PQtR) is a reinforcement learning signal that incentivizes MLLMs to generate stepwise diagnostic reasoning aligned with expert cardiac templates.
• It employs a precise mathematical formulation to balance output length against a canonical Cardiac Reasoning Template, discouraging both brevity and verbosity.
• Empirical studies in the CardiacMind framework show that PQtR enhances reasoning quality and F1 scores in echocardiographic interpretation tasks.

Procedural Quantity Reward (PQtR) is a custom reinforcement learning reward function introduced within the CardiacMind framework to induce multimodal LLMs (MLLMs) to generate more granular and physician-like diagnostic reasoning chains in echocardiographic interpretation tasks. By explicitly quantifying the match between the number of discrete reasoning steps a model outputs and a corresponding physician-curated Cardiac Reasoning Template (CRT), PQtR acts as a targeted incentive for detailed, stepwise diagnostic logic, discouraging the tendency of such models to collapse into short, under-explained answers. PQtR is mathematically formalized to provide positive signal for reasoning chains that align in length with expert templates, while preventing excessive verbosity and ensuring adherence to clinically validated reasoning structures (Qin et al., 13 Jan 2026).

1. Functional Definition and Motivation

PQtR operates as one of three novel reward signals (alongside Procedural Quality Reward, PQlR, and Echocardiographic Semantic Reward, ESR) in CardiacMind’s reinforcement learning regime. The primary goal of PQtR is to monitor and steer the quantity of reasoning steps the model outputs. In applied diagnostic settings, MLLMs frequently default to very brief answers when trained solely with accuracy-driven or likelihood-based objectives, often omitting essential logical connections. PQtR incentivizes the model to produce a number of distinct reasoning steps closely matching the expert-validated CRT, explicitly penalizing both excessive brevity and length, thereby promoting a “stepwise” diagnostic approach reflective of cardiologist workflows.

2. Mathematical Formalization

The reward computation is formalized as follows. For a diagnostic instance:

• $R = \{ h_1, \ldots, h_{|R|} \}$ is the chain of reasoning steps output by the model,
• $T$ is the corresponding CRT, with $|T|$ canonical steps,
• $\epsilon$ denotes a “verbosity tolerance” (set to 5 in experiments).

The PQtR scalar is computed as:

$$\sigma_{\text{PQtR}} = \begin{cases} \min\left(1.0, \frac{|R|}{|T|}\right), & \text{if } |R| \leq |T| + \epsilon \ 0, & \text{otherwise} \end{cases}$$

Here, the minimum with 1.0 clips overlong chains to prevent superfluous elaboration, while exceeding the template length by more than $\epsilon$ incurs zero reward. This quantification yields a direct, tunable mechanism for incentivizing alignment with clinical reasoning protocols.

3. Procedural Effect and Theoretical Rationale

Empirical and theoretical analysis underscores that PQtR enables a substantive increase in reasoning path detail by rewarding each additional, distinct procedural step up to the expert template length. In the absence of such reward shaping, MLLM outputs typically favor underspecified, high-level conclusions to avoid risk under accuracy-based optimization. By construction, PQtR ensures that reasoning sequences expand to the template target, thus preventing premature summarization. The step-count ratio structure of PQtR strikes an explicit balance between granularity and focus; chains longer than the CRT (within tolerance) are permitted but not over-rewarded, while over-verbosity (> $|T| + \epsilon$) is curtailed.

4. Integration with Multi-Objective Reward Schemes

PQtR is integrated in CardiacMind alongside PQlR—which assesses content relevance against CRT-derived canonical questions—and ESR, which enforces faithful visual grounding via CLIP-style similarity metrics. The composite per-rollout reward is given by:

$$\sigma = \lambda_{\text{format}} \cdot \sigma_{\text{format}} + \lambda_{\text{acc}} \cdot \sigma_{\text{acc}} + \delta ( \lambda_{\text{PQlR}} \cdot \sigma_{\text{PQlR}} + \lambda_{\text{PQtR}} \cdot \sigma_{\text{PQtR}} + \lambda_{\text{ESR}} \cdot \sigma_{\text{ESR}} )$$

where $\delta = 1$ only if the final answer is correct (enforcing a hallucination-reduction mechanism) and $\lambda$ coefficients are hand-tuned weights per reward. Training employs a two-stage regime: initial focus on PQtR alone for instilling stepwise structure, and subsequent optimization with the full reward mixture to refine content and grounding.

5. Empirical Outcomes and Quantitative Impact

Ablation studies reported in CardiacMind’s development demonstrate that PQtR alone, when combined with in-context CRT, increases average reasoning chain length and underlying reasoning quality. Specifically, with only PQtR enabled, the model exhibited an increase in Reasoning Quality (RQ) from 3.48 to 3.78 (on a 5-point scale) and an F1 improvement from 0.78 to 0.79 on the EchoComplex benchmark. The most pronounced gain in detailedness and granularity was attributable to PQtR; further inclusion of PQlR and ESR yielded incremental gains, but the foundational expansion of stepwise logic is credited to PQtR.

6. Comparison with Complementary Rewards (PQlR, ESR)

PQtR is distinctive among CardiacMind’s reward signals in its exclusive focus on the quantity of procedural logic, independent of content correctness or semantic grounding. PQlR enforces step-level content alignment to CRT-derived questions, while ESR addresses the linkage between textual reasoning steps and visual features in echocardiographic data. Crucially, PQlR and ESR do not penalize brevity; without PQtR, models converge on very concise high-scoring answers, undermining interpretability and transparency. By compelling an appropriate sequence length, PQtR establishes prerequisite structure for PQlR and ESR to subsequently operate at per-step fidelity.

7. Broader Significance and Limitations

The introduction of PQtR highlights the utility of reward engineering for aligning MLLM reasoning patterns with expert cognitive workflows in domains where interpretability is critical. Its procedural quantity emphasis is particularly relevant in clinical diagnostic contexts requiring stepwise, auditable logic chains. A plausible implication is that analogous rewards could be formulated for other specialized domains with well-defined procedural templates, pending further investigation. Limitations include strict dependence on the existence of high-quality CRTs and potential adaptation needs for domains with more variable stepwise structure (Qin et al., 13 Jan 2026).