On Uncertainty In Natural Language Processing (2410.03446v1)

Abstract: The last decade in deep learning has brought on increasingly capable systems that are deployed on a wide variety of applications. In natural language processing, the field has been transformed by a number of breakthroughs including LLMs, which are used in increasingly many user-facing applications. In order to reap the benefits of this technology and reduce potential harms, it is important to quantify the reliability of model predictions and the uncertainties that shroud their development. This thesis studies how uncertainty in natural language processing can be characterized from a linguistic, statistical and neural perspective, and how it can be reduced and quantified through the design of the experimental pipeline. We further explore uncertainty quantification in modeling by theoretically and empirically investigating the effect of inductive model biases in text classification tasks. The corresponding experiments include data for three different languages (Danish, English and Finnish) and tasks as well as a large set of different uncertainty quantification approaches. Additionally, we propose a method for calibrated sampling in natural language generation based on non-exchangeable conformal prediction, which provides tighter token sets with better coverage of the actual continuation. Lastly, we develop an approach to quantify confidence in large black-box LLMs using auxiliary predictors, where the confidence is predicted from the input to and generated output text of the target model alone.

Non-Exchangeable Conformal Prediction for Natural Language Generation

In the field of Natural Language Generation (NLG), the flexibility and diversity of output are crucial, albeit challenging to manage. Traditional generation models, such as those in machine translation and LLMing, provide distributions over potential next tokens at each step. However, due to the paraphrastic nature of language, determining the optimal cutoff beyond which tokens should not be considered becomes non-trivial.

Overview and Problem

This paper introduces a method for creating calibrated prediction sets for token generation, leveraging conformal prediction adjusted to handle the non-exchangeability inherent in language generation processes. The conformal prediction framework provides statistically backed guarantees about the credibility of token subsets used at each step of generation, ensuring that the model's predictions remain contextually sound even in the face of data that deviate from the training distribution.

Non-Exchangeable Conformal Prediction

Traditional conformal prediction assumes an independent and identically distributed (i.i.d.) setting, which doesn't hold in the autoregressive setup of NLG models. Here, every token's generation is contingent on previously generated tokens, breaking i.i.d. assumptions. To manage this, the approach in this paper draws on non-exchangeable conformal prediction principles, introducing a dynamic calibration mechanism through nearest-neighbor search in latent spaces.

The key is utilizing non-conformity scores from dynamically generated calibration sets derived in real-time through embeddings or latent representations. The method continuously adjusts quantile thresholds based on statistical similarities between current contexts and calibration instances, thus maintaining reliability in the generated sequences.

Application and Evaluation

LLMs and Machine Translation

The presented technique is evaluated in the setting of autoregressive LLMing and machine translation. Standard sampling techniques often fall short when the underlying data distribution shifts or in domains where a model's training doesn't perfectly align with real-world usage scenarios. This method's ability to provide tight, conformal-backed prediction sets offers an edge, achieving improved balance between exploration (diversity in generation) and exploitation (sticking to high-probability paths).

Robustness Under Distributional Shifts

Another critical investigation in this work is how the proposed method maintains efficacy when the model's latent representations are artificially corrupted to simulate distribution shifts. The results affirm the resilience of the method's calibrated sampling, which adapts effectively without significant degradation of coverage, compared to more conventional approaches.

Implications and Future Directions

This research points towards a number of implications for both theoretical exploration and practical implementations in NLG:

• Improved Generation Accuracy: By leveraging the proposed prediction sets, models can be both more diverse and accurate in their output, avoiding overly deterministic generation which often plagues greedy decoding strategies.
• Adaptive Systems: The dynamic calibration mechanism suggests applications in adaptive systems that require real-time adjustments and robust performance over a wide range of possible input distributions.
• Further Improvement and Extensions: Future work can explore extensions of this framework to non-autoregressive settings or integrate such methods with more sophisticated architectures, potentially synergizing with advancements in neural network outputs interpretability.

Overall, this method represents a meaningful progression towards better uncertainty management in language generation systems, offering significant promise for applications in varied AI domains, including conversational agents and automated content creation systems.