Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs

Abstract: LLMs generate text by sampling the next token from a probability distribution over the vocabulary at each decoding step. Popular sampling methods like top-p (nucleus sampling) often struggle to balance quality and diversity, especially at higher temperatures which lead to incoherent or repetitive outputs. We propose min-p sampling, a dynamic truncation method that adjusts the sampling threshold based on the model's confidence by using the top token's probability as a scaling factor. Our experiments on benchmarks including GPQA, GSM8K, and AlpacaEval Creative Writing show that min-p sampling improves both the quality and diversity of generated text across different model families (Mistral and Llama 3) and model sizes (1B to 123B parameters), especially at higher temperatures. Human evaluations further show a clear preference for min-p sampling, in both text quality and creativity. Min-p sampling has been adopted by popular open-source LLM frameworks, including Hugging Face Transformers, VLLM, and many others, highlighting its considerable impact on improving text generation quality.

• The paper introduces the min-p sampling method that dynamically truncates low-probability tokens to enhance both creativity and coherence.
• The approach outperforms top-p sampling on benchmarks like GPQA and GSM8K, reducing accuracy trade-offs at higher temperatures.
• Its robust performance in reasoning and creative writing tests highlights potential applications in automated content creation and conversational AI.

Min P Sampling: Balancing Creativity and Coherence at High Temperature

The paper "Min P Sampling: Balancing Creativity and Coherence at High Temperature" presents a novel approach to addressing the inherent trade-off between creativity and coherence in LLM text generation. Traditional sampling methods, such as top-$p$ sampling, often struggle to find a balance, particularly when utilizing higher temperatures. This research introduces min-$p$ sampling, a dynamic truncation method designed to enhance text coherence and creativity.

Key Contributions

Min-$p$ Sampling Method: The core innovation of this paper is min-$p$ sampling, which dynamically adjusts a minimum probability threshold for token selection based on the probability of the top candidate token. This approach aims to preserve coherence while enhancing creativity at higher temperatures by truncating the long tail of less probable tokens.

Comparative Analysis: The research provides a comprehensive comparison between min-$p$ sampling and existing sampling techniques like top-$p$. Experiments conducted across benchmarks, such as GPQA (Google-Proof QA) for reasoning, GSM8K for grade-school mathematics, and creative writing tests like AlpacaEval, demonstrate min-$p$'s superiority in producing coherent and diverse text at elevated temperatures.

Results and Implications: Min-$p$ sampling maintains or improves performance compared to top-$p$ sampling, particularly when tasked with reasoning and multi-step challenges. Results show min-$p$'s effectiveness in reducing accuracy trade-offs associated with high-temperature settings, thus allowing LLMs to generate diverse yet coherent outputs.

Experimental Insights

1. Reasoning Tasks: On GPQA and GSM8K benchmarks, min-$p$ sampling showed reduced degradation in performance compared to top-$p$, confirming its robustness in handling factual and logical tasks, even at elevated temperature levels.
2. Creative Writing: Evaluations on creative writing tasks reveal that min-$p$ sampling enhances creativity without compromising coherence, reinforcing its utility in generating high-quality text in open-ended scenarios.
3. Adoption and Validation: The paper highlights the practical utility of min-$p$ sampling, evidenced by its rapid adoption within the open-source LLM community.

Theoretical and Practical Implications

From a theoretical perspective, min-$p$ sampling offers insights into balancing stochastic processes in text generation. Practically, its application can span diverse domains requiring different creativity-coherence balances, such as storytelling, automated content creation, and conversational AI.

Future Directions

The study acknowledges limitations related to the scope of model architectures and evaluation datasets. Future work could explore min-$p$ sampling's applicability across varied models and tasks, enhancing its robustness and generalizability. Theoretical explorations into the dynamics of min-$p$ could further refine sampling strategies for LLMs.

Conclusion

Min-$p$ sampling emerges as a significant advancement in the toolkit for LLMs, particularly in scenarios demanding a nuanced balance between creativity and coherence. Its promise lies in effectively unlocking the potential of high-temperature settings, providing a user-friendly and computationally efficient alternative to existing sampling methods.