Large language model validity via enhanced conformal prediction methods

Abstract: We develop new conformal inference methods for obtaining validity guarantees on the output of LLMs. Prior work in conformal language modeling identifies a subset of the text that satisfies a high-probability guarantee of correctness. These methods work by filtering claims from the LLM's original response if a scoring function evaluated on the claim fails to exceed a threshold calibrated via split conformal prediction. Existing methods in this area suffer from two deficiencies. First, the guarantee stated is not conditionally valid. The trustworthiness of the filtering step may vary based on the topic of the response. Second, because the scoring function is imperfect, the filtering step can remove many valuable and accurate claims. We address both of these challenges via two new conformal methods. First, we generalize the conditional conformal procedure of Gibbs et al. (2023) in order to adaptively issue weaker guarantees when they are required to preserve the utility of the output. Second, we show how to systematically improve the quality of the scoring function via a novel algorithm for differentiating through the conditional conformal procedure. We demonstrate the efficacy of our approach on biography and medical question-answering datasets.

• The paper introduces conditional boosting and level-adaptive conformal prediction to address conditional validity gaps and excessive claim removal in LLM outputs.
• Conditional boosting refines claim-scoring functions, increasing retention from 24% to 39% and preserving more valuable predictions.
• Level-adaptive conformal prediction adjusts confidence thresholds per prompt, ensuring calibrated, reliable outputs for high-stakes applications.

LLM Validity via Enhanced Conformal Prediction Methods

The paper "LLM validity via enhanced conformal prediction methods" by John J. Cherian, Isaac Gibbs, and Emmanuel J. Candès presents novel techniques for improving the reliability of LLMs through advanced conformal inference methods. The authors address two primary deficiencies in existing conformal prediction approaches: conditional validity and preserving valuable claims.

Conformal prediction provides a versatile framework for uncertainty quantification in various machine learning contexts, transforming model predictions into sets with guaranteed coverage probabilities. While previous works have adapted conformal inference for LLMs by filtering out low-confidence claims, these methods do not offer conditional validity guarantees and often result in significant information loss. This paper proposes two innovative methods to mitigate these shortcomings: conditional boosting and level-adaptive conformal prediction.

Conditional Boosting

Conditional boosting aims to optimize claim-scoring functions to enhance the fidelity and utility of LLM outputs. Traditional scoring functions are not perfectly correlated with ground truths, leading to the excessive removal of accurate claims. The conditional boosting method differentiates through the conditional conformal algorithm to improve these scores systematically.

Theoretically, the authors demonstrate that by extending the framework of \citet{gibbs2023conformal}, the conditional conformal method can adaptively produce higher-quality scores. By utilizing $\beta_0 + \sum_{i=1}^3 \beta_i v^i$, with $v$ representing metadata features such as page view counts, this method retains more claims while ensuring validity. Notably, empirical results depicted in Figure \ref{fig:alpha_calibration} show that boosted scores achieve a mean claim retention of 39%, compared to 24% with unboosted scores.

Level-Adaptive Conformal Prediction

The second contribution is level-adaptive conformal prediction, which allows the confidence level, $\alpha$, to vary based on the conditions of each input prompt. This method adapts the error probability threshold to ensure outputs are valuable while maintaining calibration.

To implement this, the authors fit $\alpha(\cdot)$ functions using regression on the conformity scores, accounting for features such as prompt and response lengths. The results reported in Figure \ref{fig:alpha_calibration} validate that the adaptive $\alpha(\cdot)$ function maintains empirical calibration, providing accurate conditional guarantees while retaining a larger fraction of the LLM output.

Implications and Future Work

The advancements presented in this paper are significant for deploying LLMs in high-stakes applications such as legal assistance, healthcare, and customer service. By addressing both conditional validity and utility preservation, the proposed methods enhance the trustworthiness of LLM outputs.

Practically, these improvements mean that systems can provide users with more reliable information while minimizing the risk of propagating erroneous claims. Theoretically, this work contributes to the broader field of conformal prediction by demonstrating that conditional guarantees can be effectively integrated into complex, real-world applications.

Future research may explore further optimizations of the scoring functions, potentially incorporating more sophisticated machine learning models and additional contextual features. Additionally, expanding these methods to other forms of generative models and different domains could provide broader insights into achieving enhanced model reliability across various AI applications.

Through these contributions, Cherian, Gibbs, and Candès set a foundation for more dependable and practical applications of LLMs, bridging a crucial gap between theoretical guarantees and real-world deployment.