Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detection (2305.13282v1)

Abstract: Out-of-distribution (OOD) detection is a critical task for reliable predictions over text. Fine-tuning with pre-trained LLMs has been a de facto procedure to derive OOD detectors with respect to in-distribution (ID) data. Despite its common use, the understanding of the role of fine-tuning and its necessity for OOD detection is largely unexplored. In this paper, we raise the question: is fine-tuning necessary for OOD detection? We present a study investigating the efficacy of directly leveraging pre-trained LLMs for OOD detection, without any model fine-tuning on the ID data. We compare the approach with several competitive fine-tuning objectives, and offer new insights under various types of distributional shifts. Extensive evaluations on 8 diverse ID-OOD dataset pairs demonstrate near-perfect OOD detection performance (with 0% FPR95 in many cases), strongly outperforming its fine-tuned counterparts. We show that using distance-based detection methods, pre-trained LLMs are near-perfect OOD detectors when the distribution shift involves a domain change. Furthermore, we study the effect of fine-tuning on OOD detection and identify how to balance ID accuracy with OOD detection performance. Our code is publically available at https://github.com/Uppaal/lm-ood.

Investigating the Necessity of Fine-Tuning for OOD Detection in Pre-Trained LLMs

Introduction

Out-of-distribution (OOD) detection in NLP is a critical component for deploying trustworthy AI systems. With the ascendancy of pre-trained LLMs (PLMs) in a plethora of NLP tasks, their utility for OOD detection has become a topic of increasing importance. Traditional approaches have leaned heavily towards fine-tuning these models on in-distribution (ID) data to enhance OOD detection capabilities. However, the necessity and impact of fine-tuning for OOD detection remain largely under-investigated. In this context, we explore a fundamental question: Is fine-tuning necessary for effective OOD detection?

This paper presents a comprehensive evaluation of using PLMs directly, without any fine-tuning, for OOD detection. By employing distance-based metrics on embeddings generated by PLMs, we juxtapose the OOD detection performance of zero-shot models against their fine-tuned counterparts across several dataset pairs representing various types of distributional shifts.

Methodology

Zero-Shot OOD Detection with Pre-Trained Models

Our approach leverages the embeddings from pre-trained LLMs, hypothesizing that these representations naturally encode sufficient information to distinguish between ID and OOD samples. We employ distance-based metrics, specifically Mahalanobis distance and nearest neighbors, to measure the dissimilarity between test samples and the ID data cluster in the embedding space.

OOD Detection with Model Fine-Tuning

We further explore the impact of fine-tuning PLMs on OOD detection performance. The paper considers three fine-tuning objectives:

• Cross-Entropy Loss (CE): A commonly used loss for training neural networks.
• Task-Adaptive Pretraining (TAPT): Pretraining the model on ID data using a masked LLM objective before fine-tuning.
• Supervised Contrastive Learning (SupCon): A contrastive loss that utilizes label information to bring samples of the same class closer in the representation space.

OOD detection in the fine-tuned scenario uses the same method as in the zero-shot case but applies it to embeddings generated by the fine-tuned models.

Experimental Setup

The experimental framework involves eight diverse ID-OOD dataset pairs to ensure broad coverage of semantic and background shifts. We use RoBERTa as our pre-trained LLM given its known robustness and effectiveness across various NLP tasks. The evaluation metrics include AUROC, AUPR, FPR95, and ID classification accuracy.

Results and Analysis

Pre-Trained Models for OOD Detection

Our findings indicate that pre-trained models can serve as near-perfect OOD detectors, especially in cases of out-of-domain shifts. For instance, the zero-shot detection model achieves 0% FPR95 across several dataset pairs, outperforming fine-tuned models substantially.

Fine-Tuning and Its Impact on OOD Detection

Contrary to zero-shot models, fine-tuning tends to degrade OOD detection performance, particularly as the fine-tuning process progresses. This degradation suggests an inverse relationship between ID classification accuracy and OOD detection efficacy in fine-tuned models. However, early stopping emerges as a viable strategy for balancing the trade-off between ID accuracy and OOD detection.

Insights on Embedding Characteristics

Analysis of embeddings from pre-trained and fine-tuned models reveals that pre-trained models tend to form distinct domain clusters, facilitating effective OOD detection. Contrastingly, fine-tuning encourages models to create class-specific clusters, thereby diluting the separability between ID and OOD data.

Concluding Remarks

This paper challenges the prevailing notion that fine-tuning PLMs is essential for enhancing OOD detection capabilities. Our research underscores the potential of zero-shot models, powered by pre-trained LLMs, to deliver superior OOD detection performance in the face of out-of-domain shifts. The insights garnered from this investigation shed light on the nuanced dynamics between fine-tuning, model embeddings, and OOD detection, paving the way for future explorations aimed at optimizing the deployment of PLMs in real-world applications.