A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Code Smell Detection (2412.13801v2)

Abstract: Code smells are suboptimal coding practices that negatively impact the quality of software systems. Existing detection methods, relying on heuristics or Machine Learning (ML) and Deep Learning (DL) techniques, often face limitations such as unsatisfactory performance. Parameter-Efficient Fine-Tuning (PEFT) methods have emerged as a resource-efficient approach for adapting LLMs to specific tasks, but their effectiveness for code smell detection remains underexplored. In this regard, this study evaluates state-of-the-art PEFT methods on both Small (SLMs) and LLMs for detecting four types of code smells: Complex Conditional, Complex Method, Feature Envy, and Data Class. Using high-quality and balanced datasets sourced from GitHub, we fine-tuned four SLMs and five LLMs with PEFT techniques, including prompt tuning, prefix tuning, LoRA, and (IA)3. Results show that PEFT methods achieve comparable or better performance than full fine-tuning while consuming less GPU memory. LLMs generally outperform SLMs on detecting certain smells (e.g., Complex Conditional), while SLMs do better on others (e.g., Data Class). Additionally, increasing training dataset size significantly boosted performance, while increasing trainable parameters did not. Our findings highlight PEFT methods as effective and scalable solutions, outperforming existing heuristic-based, DL-based, and In-Context Learning approaches for code smell detection.