Evidence Markets

Abstract: Modern prediction markets face two limitations that restrict their applicability in a range of settings:~(i)~they reveal what the crowd believes but not the evidence or reasoning behind those beliefs, and~(ii)~they require an event with an external ground truth that resolves at a known future date. We address these twin challenges by introducing evidence markets, a generalization of prediction markets that incentivizes the submission of evidence alongside beliefs and can be endogenously resolved using the crowd-sourced evidence if external resolution is not possible. At its core, the market uses a logarithmic market scoring rule whose liquidity parameter changes dynamically with the accumulated evidence quality. We prove that platform loss is bounded, evidence is rewarded proportional to the current market uncertainty, and can be equivalently implemented through an automated market maker. In the case where the marker resolves endogenously based on submitted evidence, we characterize how withholding evidence shifts a trader's belief about resolution and use it to prove truthful belief and evidence reporting is a always an $\varepsilon$-dominant strategy incentive compatible (DSIC) strategy. To address operational considerations, we propose evidence verification via an LLM-as-a-Judge framework with staking and give an asynchronous execution algorithm that is not bottle-necked by verification. Throughout the work, we use LLM evaluations -- determining which model is best for a given task -- as a salient and representative running example for our proposed market.

• The paper presents a market mechanism where traders submit both probabilistic beliefs and supporting evidence, generalizing LMSR for settings with ambiguous outcomes.
• It employs a quality-sensitive liquidity function that scales evidence payoff based on market uncertainty, ensuring incentive-compatible reporting.
• The framework offers exogenous and endogenous resolution protocols with LLM-based evidence verification to maintain bounded market losses.

Evidence Markets: Integrating Belief and Evidence in Market-Based Aggregation

Introduction and Motivation

"Evidence Markets" (2606.07434) proposes a market mechanism that extends classical prediction markets by simultaneously eliciting traders' beliefs and the underlying evidence or rationale supporting those beliefs. The central innovations address two key limitations of existing prediction markets: (i) their inability to surface or incentivize explicit evidence behind beliefs, and (ii) their dependence on exogenous, time-bound, and externally verifiable outcomes. The design allows resolution to be determined either exogenously or, crucially, endogenously based on crowd-submitted evidence, with theoretical guarantees on truthful reporting and bounded loss.

This approach is especially salient for problems like LLM evaluation, scientific validation, or artifact comparison, where outcomes are determined less by objective external events and more by aggregated analysis or judgment of evidence. The mechanism generalizes the logarithmic market scoring rule (LMSR), dynamically adjusting liquidity as evidence quality accumulates, and can be equivalently cast as an automated market maker (AMM) with a quality-sensitive cost function.

Figure 1: High-level flow of the proposed evidence market, showing trader arrivals, evidence/belief submission, and resolution mechanics.

Core Market Mechanism

Evidence-Augmented LMSR and AMM

At the core of the proposed market is a generalization of LMSR: liquidity is no longer fixed but becomes a monotonic, decreasing function $\beta(R)$ of cumulative evidence quality $R$. Upon arrival, each trader submits:

• A probabilistic belief $q_t$ over $n$ possible outcomes
• An evidence set $E_t$ (possibly empty), with each element an "atomic unit" (such as an LLM evaluation Q/A pair)

Submitted evidence increases the publicly tracked cumulative evidence quality, $R_t = r(E_t^{total})$, using an application-defined, monotonic scoring function $r(\cdot)$. The dynamic liquidity means information is rewarded more when uncertainty and entropy are high, and less when high-quality evidence has already been aggregated.

The payoff mechanism for direct belief elicitation can be stated as:

$$\text{payoff}_t = \beta(R_t) \log \hat{q}_t^{\omega} - \beta(R_{t-1}) \log q_{t-1}^{\omega}$$

where $\omega$ is the realized outcome. Traders are incentivized to submit both their true belief and full available evidence as $\varepsilon$-DSIC (incentive compatible up to arbitrarily small $R$0 for endogenous resolution, exactly DSIC for exogenous).

Moreover, the mechanism can be reformulated as an AMM with a generalized cost function $R$1, where $R$2 is the vector of outstanding shares. The marginal price of evidence is strictly negative for decreasing $R$3 and scales with market entropy, meaning evidence is most valuable when collective uncertainty is largest.

Figure 2: Log scoring curves under varying liquidity parameters, showing the impact of evidence submission on both belief reporting and liquidity in the mechanism.

Resolution Protocols

• Exogenous: The market resolves when an external event's outcome is revealed (traditional case).
• Endogenous: Resolution is triggered when $R$4 distinct evidence items have been submitted. A softmax over per-alternative support determines outcome selection—a function of discriminative evidence. Theoretical results show that the effect of withholding evidence on a trader's resolution belief can be bounded arbitrarily by tuning the softmax temperature $R$5 relative to total market evidence, limiting incentives for strategic evidence manipulation.

Payoff Decomposition and Incentive Guarantees

The platform's expected loss remains bounded by $R$6, analogous to classical LMSR. Traders' rewards decompose into two parts:

1. Belief payoff: Proportional to KL divergence between reported and previous beliefs, scaled by previous liquidity.
2. Evidence payoff: Proportional to the entropy of the submitted belief, scaled by the marginal drop in liquidity from added evidence.

Risk-averse traders may receive strictly non-negative payouts purely for evidence submission, independent of directional bets. This maintains strong incentives for informationally rich evidence even from participants uninterested in risk exposure.

Evidence Verification Mechanisms

Evidence quality $R$7, central to both payout and resolution, requires robust verification. The work proposes LLM-as-a-Judge with staked disputes:

• Endogenous resolution: An LLM verifies each piece of evidence for discriminativeness, validity, and non-duplication. Disputes are possible via economic staking, echoing proof-of-stake and optimistic roll-up paradigms. Subsequent re-adjudication by the LLM, with access to dispute context, settles the issue, penalizing dishonest disputers by slashing their stake.
• Exogenous resolution: Verification is lighter. The LLM computes the marginal impact of submitted evidence on its own posteriors; evidence uninformative to the LLM accrues negligible payoff.

These approaches sidestep limitations of peer-prediction, which cannot guarantee incentive compatibility when traders may have trading and verifying roles.

Algorithmic Aspects and Execution Concerns

Practical market operation demands asynchronous trade execution due to non-instantaneous evidence verification. The authors develop a two-stage protocol:

• Pessimistic immediate execution: Traders are charged the worst-case cost (relative to possible evidence verification outcomes) at order submission.
• Verification/refunds: Once evidence is adjudicated, the platform issues rebates to traders, ensuring that all incentive and loss bounds are maintained, and immediate trading liquidity is available even with asynchronous verification.

Numerical and Analytical Results

Key analytical results include:

• Bounded Market Loss: Platform loss is upper-bounded as in classical LMSR, despite incentive-compatible evidence payouts.
• DSIC & $R$8-DSIC: Strict DSIC for beliefs and evidence in exogenous settings, and arbitrarily close to DSIC (by tuning temperature) in the endogenous case. Even large "evidence whales" can have their influence bounded.
• Evidence Value Scaling: Evidence receives higher payoff in high-entropy, uncertain states; as liquidity parameter $R$9 falls with quality, marginal evidence value decreases.

Implications and Future Directions

The proposed mechanism significantly expands the applicability of market-based information aggregation, particularly to problem domains where ground truth is not cleanly exogenous or time-bound, such as subjective, scientific, or peer-judged contest settings. Practical implementation in real-money and adversarial environments remains challenging, especially in evidence verification robustness and incentivization for costly evidence creation.

Potential future directions include:

• Developing more robust, decentralized, and tamper-resistant verification schemes, potentially involving distributed consensus among LLMs or trusted-party committees.
• Modeling and incorporating explicit effort or acquisition cost for producing evidence, further aligning economic incentives.
• Adapting the core mechanism to operate on limit order books, which dominate current prediction markets like Kalshi and Polymarket.
• Applying evidence markets to LLM evaluations at scale, replicability in science, and complex multi-agent judgment aggregation.

Conclusion

"Evidence Markets" (2606.07434) offers a unified framework incentivizing both probabilistic belief aggregation and the transparent surfacing of supporting evidence, successfully generalizing prediction markets to settings where outcomes are adjudicated by judgment rather than temporal revelation. The framework provides strong theoretical incentive guarantees, practical market implementation strategies, and robust approaches to evidence verification, paving the way for rich evidence-aware information markets in domains previously inaccessible to traditional prediction market design.