Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models (2505.03469v2)

Abstract: Recent advances in LLMs have demonstrated that Supervised Fine-Tuning (SFT) with Chain-of-Thought (CoT) reasoning data distilled from large reasoning models (e.g., DeepSeek R1) can effectively transfer reasoning capabilities to non-reasoning models. However, models fine-tuned with this approach inherit the "overthinking" problem from teacher models, producing verbose and redundant reasoning chains during inference. To address this challenge, we propose Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning (LS-Mixture SFT), which combines long CoT reasoning dataset with their short counterparts obtained through structure-preserved rewriting. Our experiments demonstrate that models trained using the LS-Mixture SFT method, compared to those trained with direct SFT, achieved an average accuracy improvement of 2.3% across various benchmarks while substantially reducing model response length by approximately 47.61%. This work offers an approach to endow non-reasoning models with reasoning capabilities through supervised fine-tuning while avoiding the inherent overthinking problems inherited from teacher models, thereby enabling efficient reasoning in the fine-tuned models.

LuaLaTeX and XeLaTeX Template for *ACL Style Files

The paper presents a concise demonstration of how to employ the *ACL style files with LuaLaTeX and XeLaTeX typesetting environments. The authors focus on compatibility with multilingual text, showcasing proper rendering of Hindi and Arabic scripts, which are crucial for researchers working on linguistics and computational language processing in diverse languages. The document does not delve deeply into novel methodologies or algorithms, nor does it introduce empirical studies or experimental results. Instead, it serves to illustrate technical implementation details relevant to academic publishing in computational linguistics.

From a technical standpoint, the paper outlines practical syntax, including commands for font declaration and language invocation, and specifies how to incorporate these within the broader framework of a LaTeX document. The authors use clear examples to show how scripts are implemented and rendered, ensuring that text maintains correct typographic properties across different languages. This aspect addresses an important concern in multilingual document preparation—accurate representation and readability of complex scripts using LaTeX formatting.

The implications of this work are largely practical, aimed at streamlining the process for researchers engaged in multilingual NLP studies who seek to publish their findings using *ACL compliant formats. Adoption of these techniques can enhance the presentation quality of academic papers, particularly those including varied linguistic examples, thereby facilitating increased accessibility and comprehension by a global audience.

While the document does not provide statistical analysis or theoretical advancements, its contributions lie in the field of document preparation—a necessary aspect of academic communication. Future developments in this domain might focus on expanding support for additional languages and scripts or optimizing the typesetting process for computational efficiency. Furthermore, as NLP models continue to evolve and expand their multilingual capabilities, tools such as the ones demonstrated in this paper will continue to play a supportive role in the dissemination of research findings. This paper is therefore a pertinent reference for computational linguists and researchers who require effective presentation of multilingual text within LaTeX environments.