How to Ask for Belief Statistics without Distortion?

Abstract: Belief elicitation is ubiquitous in experiments but can distort behavior in the main tasks. We study when, and how, an experimenter can ask for a series of action-dependent belief statistics after a subject chooses an action, while incentivize truthful reports without distorting the subject's optimal action in the main experimental tasks. We first propose a novel mechanism called the Counterfactual Scoring Rule (CSR), which achieves such nondistortionary elicitation of any single belief statistic by decomposing it into supplemental action-independent statistics. In contrast, when eliciting a fixed set of belief statistics without such decomposition, we show that robust nondistortionary elicitation is achievable if and only if the questions satisfy a joint alignment condition with the task payoff. The necessity of joint alignment is established through a graph theoretical approach, while its sufficiency follows from invoking an adaptation of the Becker-DeGroot-Marschak mechanism. Our characterization applies to experiments with general task-payoff structures and belief elicitation questions.