Kelly Betting as Bayesian Model Evaluation: A Framework for Time-Updating Probabilistic Forecasts

Abstract: This paper proposes a new way of evaluating the accuracy and validity of probabilistic forecasts that change over time (such as an in-game win probability model, or an election forecast). Under this approach, each model to be evaluated is treated as a canonical Kelly bettor, and the models are pitted against each other in an iterative betting contest. The growth or decline of each model's bankroll serves as the evaluation metric. Under this approach, market consensus probabilities and implied model credibilities can be updated real time as each model updates, and do not require one to wait for the final outcome. Using a simulation model, it will be shown that this method is in general more accurate than traditional average log-loss and Brier score methods at distinguishing a correct model from an incorrect model. This Kelly approach is shown to have a direct mathematical and conceptual analogue to Bayesian inference, with bankroll serving as a proxy for Bayesian credibility.

• The paper introduces a novel Bayesian framework that applies Kelly betting to sequentially evaluate forecast models, capturing temporal dynamics and model credibility.
• It details a mathematical methodology where model bankrolls update like Bayesian posteriors, providing real-time, granular insights through iterative betting strategies.
• Simulation studies show the framework effectively penalizes volatility and late corrections, outperforming traditional metrics such as log loss and Brier score.

Kelly Betting as Bayesian Model Evaluation: Framework and Applications

Introduction

The paper "Kelly Betting as Bayesian Model Evaluation: A Framework for Time-Updating Probabilistic Forecasts" (2602.09982) presents a formalized approach for evaluating time-updating probabilistic forecasts by leveraging Kelly betting as a Bayesian mechanism. This proposal addresses the limitations and ambiguities in traditional forecast scoring methods (such as log loss and Brier score) that fail to capture the temporal and sequential aspects of dynamic model updates. It formulates model evaluation as an iterative betting contest, where each model acts as a canonical Kelly bettor. Model credibility is represented by bankroll and evolves in real time, yielding a temporally granular, interpretable, and theoretically grounded metric.

Methodology and Mathematical Framework

The core methodology involves the following steps: at each event update, all competing models issue probability forecasts. The models determine their betting exposure using the Kelly criterion, either for binary or multinomial outcomes. Bets are settled on mutually agreed odds, yielding a real-time market consensus probability. Subsequently, each model's bankroll is marked-to-market based on updated probabilities and outstanding win shares, reflecting a Bayesian posterior over the space of models.

For binary outcomes, the optimal Kelly bet fraction is generalized to accommodate pre-existing bets via:

$f = p - \frac{(1-p)}{o}\left(1+\frac{w}{b}\right)$

where $f$ is the bet fraction, $p$ is model probability, $o$ is market odds, $w$ is win shares, and $b$ is bankroll.

The market clearing probability (consensus odds) is given by:

$$\text{market probability} = \frac{\sum p_i b_i}{1-\sum p_i w_i}$$

For multinomial outcomes, the framework shifts to portfolio-based representations, and win shares for outcome $i$ are updated as:

$w'_i = \frac{p_i}{m_i} \sum_{i=1}^{n} m_i w_i$

where $p_i$ and $m_i$ represent each bettor's and consensus forecast, respectively.

Critically, bankroll dynamics have a direct interpretation in Bayesian updating: model credibility evolves analogous to Bayesian posterior, where each outcome incrementally updates the likelihood that each model is correct.

Comparative Simulation Studies

The paper conducts extensive simulation studies, contrasting the Kelly-based evaluation to log loss and Brier score under various scenarios:

• Incorrect Probabilities: When a competing model has the wrong underlying probability (e.g., 0.53 vs. true 0.50), Kelly betting is more sensitive to prescient forecasts and penalizes late corrections appropriately.
• Recency Bias: Models that inappropriately update probabilities based on recent events are rapidly downgraded by the Kelly framework, with simulation accuracy exceeding that of log loss and Brier.
• Spurious Features: Models driven by non-predictive, volatile features are also demoted effectively, whereas traditional metrics cannot distinguish unnecessary volatility.

In iterative simulations (multiple sequential contests), the Kelly framework demonstrates a marked advantage: accuracy in identifying correct models approaches or surpasses 90% after five contests and maintains superiority as contest sequence increases. The empirical superiority is quantitatively illustrated across grid analyses.

Figure 1: Model evaluation accuracy grid after 1 game, showing the scenarios where each metric is most accurate.

Figure 2: Model evaluation accuracy grid after 5 games, marking Kelly's dominance in discriminative power.

Figure 3: Model evaluation accuracy grid after 25 games, highlighting persistence of Kelly's advantage.

Figure 4: Model evaluation accuracy grid after 50 games, with Kelly outperforming log loss and Brier across almost all scenarios.

Real-Time Credibility and Practical Applications

A central strength is real-time evaluation. At any stage of the event (prior to resolution), model bankrolls can be marked-to-market, yielding Bayesian model credibility that tracks the prescience and adaptive capabilities of each model. The framework is illustrated across multiple domains:

NFL In-Game Win Probability

Empirical tests, contrasting ESPN vs. Open Source Football, show consensus win probabilities and corresponding credibility updates for models, with the Kelly-based approach yielding nuanced, time-sensitive differentiation.

Figure 5: Comparison of in-game win probabilities for Open Source Football and ESPN for the Seahawks--Eagles game on December 18, 2023.

Figure 6: Implied model credibility for Open Source Football and ESPN, evaluated in-game.

MLB Division Winner Forecasting

Division win forecasts from FiveThirtyEight and FanGraphs are evaluated over the course of a season, with Kelly credibility dynamically tracking model prescience and adjusting prior beliefs based on early, accurate signals.

Figure 7: Weekly division win probabilities for each team, shown separately for FanGraphs and FiveThirtyEight.

Figure 8: Market consensus division win probabilities by week for the 2022 NL East Division.

Figure 9: Weekly model credibility for FanGraphs and FiveThirtyEight, based on the 2022 NL East division race.

NBA Playoff In-Game Probabilities

Kelly bankroll is propagated sequentially over dozens of NBA games, with credibility shifting in response to the granular forecast accuracy. Specific high-variance events induce large credibility shifts, exemplifying the Bayesian updating process.

Figure 10: In-game credibility for the FiveThirtyEight and Inpredictable in-game probability models, calculated sequentially over the course of the 2023 NBA playoffs.

Theoretical Implications and Connections

The formalism connects the Kelly approach to Bayesian model averaging: bankroll serves as model plausibility, sequential events provide incremental evidence, and market consensus odds synchronize agent beliefs. The paper also establishes a mathematical analogy to the PageRank algorithm, where credibility corresponds to the stationary distribution of a Markov chain defined by mutual model evaluation.

Advantages, Limitations, and Future Directions

The method possesses distinct advantages:

• Temporal Sensitivity: Rewards early, accurate forecasts and penalizes volatility.
• Real-Time Evaluation: Enables immediate model comparison without waiting for outcome resolution.
• Bayesian Interpretation: Direct mathematical concordance with Bayesian inference.
• Practical Plausibility: Easily interpretable as "would you win money betting this model?"
• Flexibility: Accommodates arbitrary prior credibilities and sequential updating.

Limitations include dependence on well-calibrated probability estimates and requirement for computational tractability in high-cardinality multinomial contests.

Future directions may include extension to continuous outcome spaces, integration with market-based ensemble forecasting, and robust handling of adversarial or strategic model behavior within the betting framework. Further theoretical work could formalize connections with consensus Bayesian aggregation, reinforcement learning, and information-theoretic metrics of forecast value.

Conclusion

The paper establishes a principled, theoretically justified, and empirically validated framework for evaluating time-updating probabilistic forecasts through Kelly betting interpreted in a Bayesian context. Its ability to reflect real-time model credibility, differentiate prescient from reactive modeling, and outperform traditional scoring metrics makes it a compelling tool for practitioners developing and deploying dynamic forecast models in sports, elections, and financial markets. Its Bayesian updating structure opens avenues for further research in sequential model selection, forecast ensemble design, and adaptive betting markets.