The Value of Mechanistic Priors in Sequential Decision Making

Abstract: Hybrid mechanistic models, physical priors with learned residuals, promise to reduce the data required for good decisions, but have no computable criterion to test this. We characterize the value of mechanistic priors in sequential decision-making within both asymptotic and burn-in regimes. To formalize this, we introduce the mechanistic information of a model -- the mutual information between the model's recommended policy $\hatπ$ and the true optimal policy $π^*$ -- quantified via an occupancy-weighted bias $B_μ$. In the asymptotic regime (large $N$), matched bounds reveal that Bayesian regret scales with the residual entropy $H_{\mathrm{mech}}$, delivering a theoretical sample complexity reduction of $H(μ)/H_{\mathrm{mech}}$ compared to an uninformed baseline. Furthermore, we provide a model certificate to determine empirical sample efficiency. Complementarily, in the clinically relevant burn-in regime (small $N$), we establish a lower bound on the penalty incurred by confidently wrong priors. We demonstrate both the asymptotic and burn-in bounds across 5-fluorouracil (5-FU) dosing simulations motivated by published FOLFOX pharmacokinetic data, where a hybrid prior yields large sample-efficiency gains in the burn-in regime. Finally, we contrast these grounded models with LLM priors, demonstrating that LLMs can suffer severe losses in mechanistic information, thereby motivating the exclusive use of physically-grounded priors for safety-critical applications.

• The paper introduces a computable quality certificate based on mechanistic information to quantify how well a model’s recommendations align with the optimal policy.
• The paper derives Bayesian regret and sample complexity bounds, demonstrating that using mechanistic priors can significantly reduce suboptimal trials, as shown in adaptive 5-FU dosing simulations.
• The paper validates the framework through simulations, highlighting that even conservative mechanistic models can yield up to 2.56× improvement in reducing cumulative regret compared to standard methods.

The Value of Mechanistic Priors in Sequential Decision Making

Introduction and Problem Setting

This paper addresses a central question in Bayesian adaptive control and sequential decision making: when and by how much do mechanistic priors—physical models with explicit biological or dynamical structure—reduce the data and interactions required to achieve near-optimal policies, compared to uninformed or purely data-driven priors? While hybrid methods combining mechanistic models and statistical learning underpin disciplines from scientific machine learning to adaptive medicine, there has been no computable, pre-trial quality criterion to certify whether a specific mechanistic model will deliver sample efficiency for a concrete task.

The authors introduce the mechanistic information of a model, quantifying how much the model's recommendations inform the true optimal policy, and relate this to Bayesian regret and sample complexity in both asymptotic and finite-sample (burn-in) regimes. The framework is instantiated and validated through simulations of 5-fluorouracil (5-FU) chemotherapy dosing—a clinically relevant, safety-critical domain.

Mechanistically, the setting is formalized by reducing continuous control under uncertain dynamics to a discrete policy bandit problem (multi-armed bandit: MAB), where each arm corresponds to a candidate policy drawn from a discretized policy class. The mechanistic model serves as a prior, recommending a policy by simulating reward integrals under the (potentially inaccurate) calibrated dynamics, while the ground truth is determined by the (unknown) patient-level dynamics.

Figure 1: Three layers of the hybrid model learning setting: (A) mechanistic model; (B) information flow from prior and model to policymaking and the environment; (C) regret-bias diagram, illustrating the regime switch at the critical model bias.

Mechanistic Information and Quantitative Model Certificates

The core formal contribution is the introduction of mechanistic information $(I_\mu(\pi^*;\hat{\pi}))$, the mutual information between the model’s recommended policy and the optimal policy. Under an occupancy-weighted metric, model bias $B_\mu$ is defined as the RMS gap in reward between true and modeled dynamics, weighted by the prior occupancy of each policy. Mechanistic information is then upper bounded by a Gaussian channel capacity, parameterized by the model bias, noise, and prior entropy, allowing for closed-form computable quantities from calibration or pre-trial data.

Consequently, the critical bias $B_\mu^{\mathrm{crit}}$ is derived—a threshold, computable from calibration data, that separates two regimes:

• When $B_\mu < B_\mu^{\mathrm{crit}}$, the mechanistic model can guarantee reduction in sample complexity (i.e., fewer suboptimal trials).
• When $B_\mu \geq B_\mu^{\mathrm{crit}}$, the model provides negligible gain over an uninformed baseline.

This quality certificate is robust to parameter variations, as corroborated by detailed sensitivity analyses.

Figure 2: Sensitivity of the model-quality certificate to parameters $\kappa_\mu$ (Lipschitz sensitivity), $B_\mu$ (model bias), and $d_F$ (residual GP rank). Top: Channel capacity $C(B_\mu)$ (mechanistic information upper bound) as a function of each parameter; bottom: heatmaps of $B_\mu / B_\mu^{\mathrm{crit}}$ across parameter pairs, with operational boundaries indicated.

Asymptotic Regime: Regret and Sample Complexity Bounds

Leveraging recent information-theoretic analysis of Bayesian bandits, the authors establish matched lower and upper bounds for Bayesian regret in the large-sample regime. The minimax lower bound scales as $B_\mu$0 where $B_\mu$1 is the residual entropy after accounting for mechanistic information. Thompson Sampling achieves this bound up to a $B_\mu$2 slack.

The ratio of sample complexities—uninformed versus mechanistic—thus scales as $B_\mu$3, providing a concrete operational metric for quantifying the benefit of the prior:

$B_\mu$4

Empirically, in a classical 5-FU dosing scenario ($B_\mu$5 policies, $B_\mu$6 cycles), the authors observe a certified reduction of $B_\mu$7 in the worst-case, with much larger empirical improvements in more informative model settings.

Figure 3: Critical bias as a function of $B_\mu$8 (policy class size); the blue curve plots the theoretical sample complexity ratio, with all other parameters at their literature-calibrated values.

Burn-in Regime and Robustness to Misspecification

In the finite-sample (burn-in) regime, the critical concern is overconfidence in a misspecified prior, as is highly relevant in the initial cycles of clinical treatment. A tight lower bound quantifies the regret cost of such priors, which grows logarithmically with the prior’s misplaced confidence and the required identification accuracy. For practical scenarios, this provides quantitative guarantees on minimum exploration (and regret) required before adaptation can occur. Again, the framework's predictions are verified by matching trends in empirical simulation (e.g., calibration to cycle counts observed in prospective clinical datasets).

Extending to priors derived from LLMs instead of physical mechanisms, the analysis shows that LLMs can lose the majority of their mechanistic information under modest distributional shifts, an especially acute problem in settings with few candidate arms ($B_\mu$9 small) and high-stakes mismatch between training and deployment distributions. This provides a formal argument for the exclusive or at least primary use of mechanistic (physically-grounded) priors in safety-critical applications, such as individualized dosing in oncology.

Simulation and Clinical Context

Simulations of adaptive 5-FU dosing, with comprehensive calibration to population pharmacokinetics and clinical reward structure, validate both the asymptotic and burn-in theory. Even conservative mechanistic models achieve marked gains in sample efficiency: at certified model quality thresholds, hybrid Thompson sampling reduces cumulative regret (subtherapeutic cycles) by $B_\mu^{\mathrm{crit}}$0 relative to standard-of-care (BSA dosing) and by $B_\mu^{\mathrm{crit}}$1 versus adaptive TS without a mechanistic prior.

These results are robust to uncertainties in model calibration, reward noise, or discretization granularity, as shown in broad parameter sweeps. The sample-efficiency gain is monotonic in mechanistic information, and the proposed model-quality certificate consistently separates the actionable, data-efficient regime from the baseline.

Implications, Limitations, and Future Directions

The formalism and results in this paper offer a way to operationalize the value of mechanistic priors in sequential control problems, with actionable, computable certificates for clinical (and other) adaptive trials. Practitioners can determine, before trial onset, whether a given model—calibrated on cohort data and with quantified residual—has the potential to reduce the number of suboptimal interventions required. This is particularly important in medical settings, where every cycle directly impacts patient outcomes.

Limitations:

• The analysis focuses on discrete-action versions of the control problem ($B_\mu^{\mathrm{crit}}$2 finite); continuous-control extensions, closing the gap to general dynamical systems, are left as future work.
• The ODE model calibration and residuals are based on population-level or synthetic data, not patient-level real data; clinical impact requires prospective studies.
• The results assume access to accurate estimation of model bias and entropy; incorporating the estimation error and its effect on regret is a natural extension.

Theoretical and Practical Outlook:

Future research may extend the framework to directly estimate mechanistic information from per-patient or personalized residuals; to population-level adaptive protocols exploiting pooled statistics; and to multi-stage or continuous-time control. The clear demonstration of the inferiority of LLM-derived priors under distribution shift, compared to physically-informed priors, sets a strong methodological direction for safety assurance in critical applications of AI-driven decision making.

Conclusion

This work provides, for the first time, an information-theoretically rigorous and computable framework for quantifying and certifying the value of mechanistic priors in sequential decision making. Its critical-bias-based model certificate, matched regret bounds, and empirical validation in adaptive dosing constitute a robust substrate for sample-efficient and safe adaptive policies in clinical and scientific domains. The clear trade-off between mechanistic information, model bias, and regret is both operational and predictive, guiding practitioners toward priors that are not only plausible but provably useful for efficient learning and policy deployment in real-world sequential decision problems.