Smaller Language Models Are Better Instruction Evolvers (2412.11231v1)

Abstract: Instruction tuning has been widely used to unleash the complete potential of LLMs. Notably, complex and diverse instructions are of significant importance as they can effectively align models with various downstream tasks. However, current approaches to constructing large-scale instructions predominantly favour powerful models such as GPT-4 or those with over 70 billion parameters, under the empirical presumption that such larger LLMs inherently possess enhanced capabilities. In this study, we question this prevalent assumption and conduct an in-depth exploration into the potential of smaller LLMs (SLMs) in the context of instruction evolution. Extensive experiments across three scenarios of instruction evolution reveal that smaller LLMs (SLMs) can synthesize more effective instructions than LLMs. Further analysis demonstrates that SLMs possess a broader output space during instruction evolution, resulting in more complex and diverse variants. We also observe that the existing metrics fail to focus on the impact of the instructions. Thus, we propose Instruction Complex-Aware IFD (IC-IFD), which introduces instruction complexity in the original IFD score to evaluate the effectiveness of instruction data more accurately. Our source code is available at: \href{https://github.com/HypherX/Evolution-Analysis}{https://github.com/HypherX/Evolution-Analysis}

• The paper presents evidence that smaller language models evolve more diverse and complex instructions, outperforming larger models in various tasks.
• It introduces the novel Instruction Complex-Aware IFD (IC-IFD) metric to accurately assess instruction quality without traditional tuning.
• The findings imply that cost-efficient smaller models can lower development barriers and inspire future AI research on model efficiency and instruction synthesis.

Evaluating the Efficacy of Smaller LLMs in Instruction Evolution

The paper "Smaller LLMs Are Better Instruction Evolvers" challenges the prevailing assumption that larger LLMs, such as those with parameters exceeding 70 billion, are inherently superior in synthesizing complex and diverse instructions. This paper investigates the capabilities of smaller LLMs (SLMs) in the context of instruction evolution, an area critical to maximizing the utility of AI in various downstream tasks by aligning them effectively through instruction tuning.

Key Findings and Methodology

1. Contradiction of Conventional Wisdom: The authors provide compelling evidence through extensive experiments across several instruction evolution scenarios, including Evol-Instruct, AutoIF, and Auto Evol-Instruct. The paper demonstrates that SLMs outperform LLMs in generating more complex and varied instructions. Notably, SLMs were found to have a broader output space, indicating a greater diversity in the instructions they evolved.
2. Novel Evaluation Metric - IC-IFD: A significant contribution of this paper is the introduction of the Instruction Complex-Aware IFD (IC-IFD) score. This metric enhances the original IFD score by incorporating instruction complexity, enabling a more accurate assessment of the effectiveness of instruction data without the need for instruction tuning. This advancement addresses a critical gap in current evaluation methodologies that often overlook the inherent quality of instructions themselves.
3. Numerical and Qualitative Evidence: The performance of SLM-synthesized instructions was superior in instruction following, mathematical reasoning, and code generation tasks across various models, such as Llama and Qwen series models. The paper provides detailed numerical results and explores the distribution of top-1 token probabilities, which illustrate the tendency of LLMs to produce less diverse outputs due to their stronger instruction following capabilities.
4. Adaptations and Implications: In exploring "Why do SLMs Outperform LLMs?" the authors elucidate that SLMs avoid the pitfalls of overconfidence that can narrow the output possibilities of LLMs. This results in SLMs generating a wider range of instructional variants, contributing to improved task performance. The implications suggest a potential shift in how instruction data is synthesized, favoring SLMs for reduced computational demand and enhanced instruction evolution quality.

Practical and Theoretical Implications

This research has both practical and theoretical implications. Practically, it suggests a more cost-effective approach to instruction synthesis by leveraging SLMs, which could lower the entry barrier for developing advanced AI applications. Theoretically, it challenges existing paradigms in AI models regarding size and capability, suggesting that smaller models hold unexplored potential, especially in contexts that demand innovation and variability.

Future Research Directions

Future research could explore the specific architectural features or training paradigms that grant SLMs their efficacy in instruction evolution. Additionally, expanding this research into broader AI alignment tasks or exploring other evaluation metrics could enrich the understanding of model capabilities. The openness of the paper towards smaller models invites an exciting avenue for future research to explore size and efficiency in AI models further.

In conclusion, this paper posits an impactful reevaluation of the assumed superiority of larger models in instruction evolution, offering a well-substantiated argument for the reconsideration of SLMs as viable, efficient alternatives for complex instruction synthesis.