Uncertainty in Natural Language Processing: Sources, Quantification, and Applications (2306.04459v1)

Abstract: As a main field of artificial intelligence, NLP has achieved remarkable success via deep neural networks. Plenty of NLP tasks have been addressed in a unified manner, with various tasks being associated with each other through sharing the same paradigm. However, neural networks are black boxes and rely on probability computation. Making mistakes is inevitable. Therefore, estimating the reliability and trustworthiness (in other words, uncertainty) of neural networks becomes a key research direction, which plays a crucial role in reducing models' risks and making better decisions. Therefore, in this survey, we provide a comprehensive review of uncertainty-relevant works in the NLP field. Considering the data and paradigms characteristics, we first categorize the sources of uncertainty in natural language into three types, including input, system, and output. Then, we systemically review uncertainty quantification approaches and the main applications. Finally, we discuss the challenges of uncertainty estimation in NLP and discuss potential future directions, taking into account recent trends in the field. Though there have been a few surveys about uncertainty estimation, our work is the first to review uncertainty from the NLP perspective.

The estimation and management of uncertainty are critical considerations in the deployment of deep learning models for NLP tasks. Given that neural networks often function as "black boxes" and rely on probabilistic computations, understanding the reliability of their outputs is paramount for risk mitigation and informed decision-making. The survey "Uncertainty in Natural Language Processing: Sources, Quantification, and Applications" (Hu et al., 2023 ) provides a comprehensive review of uncertainty-related research specifically within the NLP domain.

Sources of Uncertainty in NLP

The survey categorizes the origins of uncertainty in NLP into three primary types, reflecting the characteristics of natural language data and common processing paradigms (Hu et al., 2023 ):

1. Input Uncertainty: This pertains to the inherent ambiguity and variability present in natural language itself. Sources include:
   • Linguistic Ambiguity: Lexical (word sense), syntactic (parsing structure), semantic (meaning interpretation), and pragmatic (contextual intent) ambiguities are pervasive in text.
   • Noise and Perturbations: Real-world text often contains errors such as typos, grammatical mistakes, informal language, and variations introduced during data acquisition (e.g., OCR errors, ASR transcriptions).
   • Domain/Distribution Shift: Models trained on one data distribution often encounter inputs from different distributions during deployment, leading to uncertainty about their applicability. This includes shifts in topic, style, or time period.
   • Incompleteness: Input text may lack sufficient context for unambiguous interpretation.
2. System Uncertainty: This arises from the modeling process and the limitations of the model itself. It primarily corresponds to epistemic uncertainty, which reflects the model's lack of knowledge and could theoretically be reduced with more data or a better model. Sources include:
   • Model Specification: Uncertainty stemming from the choice of model architecture, hyperparameters, and inductive biases. Different models may capture different aspects of the data distribution, leading to uncertainty where their predictions diverge.
   • Parameter Uncertainty: Even with a fixed architecture, uncertainty exists regarding the optimal parameter values ($\theta$) given the finite training data $D$. Bayesian methods aim to capture the posterior distribution $p(\theta|D)$ rather than finding a single point estimate.
   • Training Process: Stochasticity in optimization algorithms (e.g., SGD), initialization, and data shuffling can lead to different models even when trained on the same data, contributing to uncertainty.
3. Output Uncertainty: This relates to the uncertainty inherent in the prediction or generation process, even if the input and model parameters were perfectly known. It often corresponds to aleatoric uncertainty, reflecting irreducible randomness or ambiguity in the underlying data generating process. Sources include:
   • Task-Intrinsic Ambiguity: Some NLP tasks are inherently multi-modal or subjective. For example, there might be multiple valid translations for a sentence, several plausible answers to a question, or diverse ways to complete a text prompt.
   • Prediction Calibration: The confidence scores produced by models (e.g., softmax probabilities) may not accurately reflect the true likelihood of correctness. Miscalibrated outputs introduce uncertainty about the reliability of high-confidence predictions.
   • Evaluation Uncertainty: Uncertainty can also arise in evaluating system outputs, especially for tasks with subjective criteria or multiple reference standards (e.g., text generation, summarization).

Understanding and distinguishing these sources is crucial for selecting appropriate quantification methods and mitigation strategies.

Uncertainty Quantification (UQ) Approaches in NLP

The survey reviews various methodologies developed for quantifying uncertainty in neural networks, adapted for or applied within the NLP context (Hu et al., 2023 ). These can be broadly grouped:

1. Bayesian Approaches: These methods explicitly model uncertainty over model parameters or functions, typically distinguishing between epistemic and aleatoric uncertainty.
   • Bayesian Neural Networks (BNNs): Place prior distributions over model weights and use techniques like Variational Inference (VI) or Markov Chain Monte Carlo (MCMC) to approximate the posterior distribution $p(\theta|D)$. Predictions involve marginalizing over the posterior: $$p(y|x, D) = \int p(y|x, \theta) p(\theta|D) d\theta$$. Predictive variance can be decomposed into epistemic and aleatoric components. Implementation often involves libraries like Pyro or TensorFlow Probability. Challenges include scalability to large transformer models and the choice/quality of priors and approximate posteriors.
   • Monte Carlo Dropout (MCD): Approximates a BNN by performing multiple stochastic forward passes ($T$ times) through a standard network with dropout enabled at test time. The predictive mean $$\hat{y} \approx \frac{1}{T} \sum_{t=1}^T f(x, \hat{\theta}_t)$$ and sample variance $$\text{Var}(\hat{y}) \approx \frac{1}{T} \sum_{t=1}^T (f(x, \hat{\theta}_t) - \hat{y})^2 + \tau^{-1} I$$ (where $\tau$ relates to model precision) serve as uncertainty estimates, primarily capturing epistemic uncertainty. It is relatively easy to implement by adding dropout layers and modifying the inference procedure.

     # Pseudocode for MCD inference def mcd_predict(model, x, num_samples=20): model.train() # Enable dropout outputs = [] for _ in range(num_samples): outputs.append(model(x)) # Example for classification: calculate mean probability and variance probabilities = torch.stack([torch.softmax(out, dim=-1) for out in outputs]) mean_prob = probabilities.mean(dim=0) predictive_entropy = -torch.sum(mean_prob * torch.log(mean_prob + 1e-10), dim=-1) # Measure of total uncertainty expected_entropy = -torch.mean(torch.sum(probabilities * torch.log(probabilities + 1e-10), dim=-1), dim=0) # Approximation of aleatoric mutual_information = predictive_entropy - expected_entropy # Approximation of epistemic return mean_prob, predictive_entropy, mutual_information

   • Deep Ensembles: Trains multiple ($M$) identical networks independently using different random initializations (and potentially data shuffling). Predictions are averaged, and the variance across ensemble members serves as an uncertainty measure: $$\text{Var}(\hat{y}) \approx \frac{1}{M} \sum_{m=1}^M (f(x, \theta_m) - \bar{f}(x))^2$$, where $\bar{f}(x) = \frac{1}{M} \sum_m f(x, \theta_m)$. Ensembles often achieve strong performance and calibration but incur significant computational cost during training and inference (M times). They primarily capture epistemic uncertainty.
2. Non-Bayesian Approaches: These methods estimate uncertainty without explicit Bayesian modeling, often focusing on calibration or OOD detection.
   • Softmax Probability Analysis: Using the maximum softmax probability (MSP) or predictive entropy $H(y|x) = -\sum_k p(y=k|x) \log p(y=k|x)$ directly as confidence scores. While simple, these are often poorly calibrated, especially for deep models.
   • Temperature Scaling: A post-hoc calibration method that learns a single scalar temperature parameter $T$ to rescale logits before the softmax: $q(y=k|x) = \frac{\exp(z_k/T)}{\sum_j \exp(z_j/T)}$. $T$ is optimized on a validation set to minimize calibration error (e.g., ECE). It primarily addresses calibration without explicitly quantifying uncertainty sources.
   • Distance-based Methods: Utilize distances in the model's embedding space (e.g., Mahalanobis distance between test sample representation and class-conditional Gaussians fitted on training data representations) to detect OOD inputs or estimate uncertainty.
   • Loss Prediction: Training an auxiliary model to predict the main model's loss or error on a given input, using this prediction as an uncertainty score.
3. Other Approaches: Methods like quantile regression or conformal prediction provide rigorous uncertainty intervals with coverage guarantees but may require specific model architectures or assumptions.

The choice of UQ method involves trade-offs between computational complexity, ease of implementation, theoretical grounding, ability to disentangle uncertainty sources, and empirical performance on specific NLP tasks.

Applications of Uncertainty in NLP

Quantified uncertainty enables numerous downstream applications aimed at creating more reliable, robust, and efficient NLP systems (Hu et al., 2023 ):

• Selective Prediction / Rejection: Systems can abstain from making predictions when uncertainty exceeds a predefined threshold, deferring to a human expert or a fallback system. This is crucial in high-risk domains like medical diagnosis or financial analysis based on text. Implementation involves setting a threshold on an uncertainty metric (e.g., predictive entropy, MCD variance, ensemble variance) and rejecting samples above it.
• Active Learning (AL): Uncertainty estimates guide the selection of the most informative unlabeled data points for annotation. Common AL strategies include selecting samples with maximum predictive entropy, highest variance (MCD/Ensembles), or using Bayesian Active Learning by Disagreement (BALD), which maximizes the mutual information between predictions and model parameters $I(y; \theta | x, D)$. This improves labeling efficiency and model performance, especially in low-resource settings.
• Out-of-Distribution (OOD) Detection: Uncertainty scores can serve as indicators of whether an input sample comes from a distribution different from the training data. High epistemic uncertainty often correlates with OOD inputs. Techniques often involve thresholding uncertainty scores derived from MCD, ensembles, or distance-based methods.
• Improving Model Robustness: Uncertainty can signal potential errors due to domain shift or adversarial perturbations. Models can be trained or adapted to be more robust by incorporating uncertainty awareness, for instance, through uncertainty-regularized training objectives.
• Enhancing Interpretability and Trustworthiness: Communicating uncertainty alongside predictions allows users to gauge the model's confidence. Visualizing uncertainty (e.g., highlighting uncertain words in a sequence) can provide insights into model reasoning and failure modes.
• Calibration: Applying UQ methods like temperature scaling, ensembles, or Bayesian approaches often leads to better-calibrated models, where confidence scores more accurately reflect the probability of correctness. Calibration is typically measured using metrics like Expected Calibration Error (ECE) or Brier score.
• Task-Specific Applications:
  • Machine Translation: Identifying low-confidence translations, potentially triggering post-editing or alternative translation generation.
  • Text Classification/NER: Detecting ambiguous classifications or entity spans.
  • Question Answering: Assessing confidence in generated answers, distinguishing between "don't know" and uncertain answers.
  • Dialogue Systems: Managing uncertainty in user intent recognition or response generation.
  • Natural Language Generation: Quantifying uncertainty in generated sequences, potentially guiding beam search or sampling strategies.

Challenges and Future Directions

Despite progress, several challenges remain in the field of uncertainty for NLP, as highlighted by the survey (Hu et al., 2023 ):

• Scalability: Applying rigorous Bayesian methods (BNNs via VI/MCMC) to state-of-the-art LLMs with billions of parameters remains computationally prohibitive. Efficient approximations like MCD or Laplace approximation are needed, but their quality needs careful assessment. Ensembles face similar scaling issues.
• Disentanglement: Reliably separating aleatoric and epistemic uncertainty is challenging but crucial, as they require different mitigation strategies (more data for epistemic, potentially model changes or acknowledging ambiguity for aleatoric). Current methods provide approximations whose quality varies.
• Calibration under Distribution Shift: Ensuring that uncertainty estimates remain calibrated when the data distribution changes is difficult. Post-hoc calibration methods like temperature scaling might not generalize well out-of-distribution.
• Evaluation Metrics: Standardized, reliable, and task-appropriate metrics for evaluating the quality of uncertainty estimates (beyond just calibration) are needed. Metrics should ideally assess the usefulness of uncertainty for downstream tasks like OOD detection or selective prediction.
• Uncertainty in Pre-trained Models: Understanding and quantifying uncertainty in massive pre-trained models (e.g., GPT-4, Llama) is an active area. How does pre-training affect uncertainty? How can uncertainty be estimated efficiently during fine-tuning or prompting?
• Structured Outputs: Extending UQ methods effectively to structured prediction tasks common in NLP (e.g., sequence generation, parsing) is more complex than classification. Defining and quantifying uncertainty over sequences or trees requires specialized techniques.
• Integration into Complex Systems: Effectively propagating and utilizing uncertainty information through multi-stage NLP pipelines remains an open challenge.

Future research is expected to focus on developing scalable UQ methods for LLMs, improving the disentanglement and calibration of uncertainty estimates, creating better evaluation protocols, and exploring novel applications that leverage uncertainty for more trustworthy and robust NLP systems.

Conclusion

Uncertainty quantification is an increasingly vital component of responsible NLP development and deployment. By identifying the sources of uncertainty inherent in language and modeling, applying appropriate quantification techniques (ranging from Bayesian methods to simpler heuristics), and leveraging this information in downstream applications, practitioners can build more reliable, robust, and trustworthy NLP systems. Addressing the current challenges, particularly concerning scalability and evaluation in the era of LLMs, remains a key focus for ongoing research.