On Subjective Uncertainty Quantification and Calibration in Natural Language Generation (2406.05213v2)

Abstract: Applications of LLMs often involve the generation of free-form responses, in which case uncertainty quantification becomes challenging. This is due to the need to identify task-specific uncertainties (e.g., about the semantics) which appears difficult to define in general cases. This work addresses these challenges from a perspective of Bayesian decision theory, starting from the assumption that our utility is characterized by a similarity measure that compares a generated response with a hypothetical true response. We discuss how this assumption enables principled quantification of the model's subjective uncertainty and its calibration. We further derive a measure for epistemic uncertainty, based on a missing data perspective and its characterization as an excess risk. The proposed methods can be applied to black-box LLMs. We illustrate the methods on question answering and machine translation tasks. Our experiments provide a principled evaluation of task-specific calibration, and demonstrate that epistemic uncertainty offers a promising deferral strategy for efficient data acquisition in in-context learning.

Analyzing Subjective Uncertainty Quantification and Calibration in LLMs

In the paper "On Subjective Uncertainty Quantification and Calibration," researchers Ziyu Wang and Chris Holmes explore the challenging domain of uncertainty quantification within free-form Natural Language Generation (NLG). They leverage Bayesian decision theory to address the intricacies of evaluating subjective uncertainties when dealing with the semantic and syntactical complexities of LLMs (LMs). Their approach emphasizes the use of utility-based similarity measures to quantify task-specific uncertainties and introduces methods to evaluate the calibration of these models. This review will provide an in-depth perspective on the key contributions, experimental results, and broader implications of their work.

Methodology Overview

The authors begin by framing the problem within a Bayesian decision-theoretic setup, ensuring the utility is defined via a task-specific similarity measure, denoted as $S(y', y; I)$. This measure captures the utility when a generated response $y'$ is evaluated against a hypothetical true response $y$ given an instruction $I$. The expected utility maximization principle underlies their approach, where the generation $y'$ aims to maximize $E_{y \sim p_M(\cdot \mid I)} S(y', y; I)$. This principle generalizes to multiple NLG tasks, including QA and machine translation.

Subjective Uncertainty Measure

The authors employ the Bayes risk framework to define subjective uncertainty, leveraging the minimum achievable risk given the model's predictive distribution $p_M$. They argue that previous methods focusing on semantic uncertainty can be adapted to this broader setup, providing a unique, principled aggregation of similarity measures among generations.

Calibration Evaluation

The calibration of subjective uncertainty measures is particularly pivotal. Calibration is assessed through a decision-theoretic lens: an LM is calibrated if its expected utility matches the actually incurred utility under the true data distribution. The authors propose utilizing reliability diagrams and a generalized version of expected calibration error (gECE) to evaluate calibration, addressing previously unresolved challenges in free-form NLG calibration.

Epistemic Uncertainty in In-Context Learning

A novel contribution of this paper is the decomposition of predictive uncertainty into epistemic and aleatoric components. The quantification of epistemic uncertainty, especially in in-context learning (ICL) scenarios, is methodologically challenging. The authors draw from a missing data perspective and define epistemic uncertainty through reducible risk, highlighting its connection to Bayesian modeling and existing literature on excess risk. This approach elucidates how epistemic uncertainty exclusively accounts for risk reducible by additional data.

Experimental Illustrations

The authors validate their methodologies by conducting experiments on free-form QA and machine translation tasks.

Free-Form QA

Using GPT-3.5, they evaluate tasks such as CoQA and NQOpen. The gECE is applied, demonstrating varying calibration levels across the tasks. Notably, the LM shows overconfidence in open-domain tasks like NQOpen, aligning with predictions about the calibration limitations for models on such tasks.

In-Context Machine Translation

In this domain, the authors leverage the FLORES+ dataset for several language pairs. The utility is defined using the chrF score, capturing both semantics and syntax. The experiments reveal that LMs display poor calibration on low-resource languages like Yue Chinese, yet demonstrate better calibration on more resource-rich languages like French. The authors also dissect epistemic uncertainties, showing strong correlation between reducible uncertainty and task performance improvements with increased ICL sample sizes.

Implications and Future Directions

This paper advances the understanding of uncertainty quantification in LMs by providing principled, decision-theoretic approaches that can generalize across various NLG tasks. The demonstrated methodologies illuminate the intricate balance between subjective uncertainty and model calibration, offering tools to diagnose and enhance LM performance. While the work is firmly rooted in theoretical foundations, it also paves the way for practical applications in LM deployment, particularly in tasks where post-calibration might not be feasible.

Future research could expand on recalibration techniques or explore the integration of these uncertainty measures within conformal prediction frameworks. Another intriguing direction would be to investigate whether LMs' verbalized uncertainties align with these principled subjective uncertainty measures. Such investigations could unveil deeper insights into aligning model predictions with human expectations and decision-making standards.

In summary, the paper by Wang and Holmes equips the research community with robust methodologies for quantifying and dissecting uncertainties in LMs, enhancing our capability to deploy these models more effectively and reliably across diverse applications.