Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies (2410.19878v2)

Published 24 Oct 2024 in cs.CL, cs.AI, and cs.LG
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies

Abstract: The large models, as predicted by scaling raw forecasts, have made groundbreaking progress in many fields, particularly in natural language generation tasks, where they have approached or even surpassed human levels. However, the unprecedented scale of their parameters brings significant computational and storage costs. These large models require substantial computational resources and GPU memory to operate. When adapting large models to specific downstream tasks, their massive parameter scale poses a significant challenge in fine-tuning on hardware platforms with limited computational power and GPU memory. To address this issue, Parameter-Efficient Fine-Tuning (PEFT) offers a practical solution by efficiently adjusting the parameters of large pre-trained models to suit various downstream tasks. Specifically, PEFT adjusts the parameters of pre-trained large models to adapt to specific tasks or domains, minimizing the introduction of additional parameters and the computational resources required. This review mainly introduces the preliminary knowledge of PEFT, the core ideas and principles of various PEFT algorithms, the applications of PEFT, and potential future research directions. By reading this review, we believe that interested parties can quickly grasp the PEFT methodology, thereby accelerating its development and innovation.

PDF HTML Abstract

An Expert Overview of "Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies"

The paper "Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies" by Wang et al. systematically explores the sophisticated landscape of parameter-efficient fine-tuning (PEFT) methods within the context of LLMs. The immense increase in the scale of model architectures, while yielding a notable enhancement in task performance, has introduced substantial computational and storage burdens due to the vast number of parameters involved. This survey comprehensively explores the mechanisms and strategic innovations designed to mitigate these challenges.

Key Insights into PEFT Techniques

The paper presents an extensive classification and analysis of various PEFT methodologies, categorizing them into several primary classifications: Additive, Reparameterized, Selective, Hybrid, Quantization, and Multi-task PEFT methods. Each class addresses the fine-tuning problem through different paradigms—by either introducing new trainable parameters, restructuring existing model parameters, or efficiently selecting parameters for training.

  • Additive PEFT techniques, such as Adapter layers and Soft Prompts, focus on adding extra modules or parameters that are adjusted during the fine-tuning process while keeping the core model parameters fixed. These approaches notably reduce the resource cost with negligible loss in model performance across diverse tasks.
  • Reparameterized PEFT endeavors involve techniques like Low-rank Decomposition (exemplified by LoRA), which structurally reorganize model parameters into a more conducive form for efficient adaptation during training.
  • Selective PEFT methods refine the focus of tuning efforts by targeting specific subsets of parameters. This strategy aligns well with the principles of sparsity and efficiency in model adaptation.

Comparative Analysis and Implications of PEFT

Through comparing existing surveys, the paper highlights that most prior work has not addressed practical applications across divergent domains comprehensively. By assembling a breadth of techniques and their applications in vision, diffusion, and multimodal models, this work lends significant depth to the academic conversation around adapting large models to constrained computational environments.

The implications of these methodologies resonate strongly in fields where computational resource constraints are critical but high accuracy is necessary, such as healthcare AI, real-time translation services, and edge computing applications. The paper suggests that advances in PEFT not only foster resource-efficient AI solutions but also expand the applicability of AI to real-world problems that necessitate fast, efficient model update capabilities.

Future Directions in PEFT Research

The authors identify promising avenues for PEFT research, such as the exploration of multi-objective learning tasks that address an array of fine-tuning challenges. Furthermore, the paper advocates for the expansion of PEFT techniques into multimodal and continual learning scenarios, which are imperative for advancing AI's capacities to handle dynamically evolving data from multiple input sources.

Concrete algorithmic strategies, like the automated design of adaptable modules (such as adapters and prompts), and enhanced heuristic strategies for hybrid PEFT methods, are underscored as critical areas needing robust exploration. This survey thus calls for greater focus on automated approaches in PEFT configurations that dynamically tailor to both text and multimodal data landscapes.

Conclusion

In summary, the survey presented by Wang et al. contributes a critical academic resource, systematically categorizing and discussing a pivotal aspect of modern AI architectures—Parameter-Efficient Fine-Tuning. It challenges the frontier of AI research by reducing computational barriers and adapts large pre-trained models more effectively across diverse tasks and environments. This resource not only serves as a guiding framework for current AI applications but also as a foundational stepping stone for authors and practitioners aiming to modularize and strategically optimize AI efforts amid the constraints inherent in real-world deployment contexts.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (241)
  1. S. Wu, H. Fei, L. Qu, W. Ji, and T.-S. Chua, “Next-gpt: Any-to-any multimodal llm,” arXiv preprint arXiv:2309.05519, 2023.
  2. J. Li, T. Tang, W. X. Zhao, J.-Y. Nie, and J.-R. Wen, “Pre-trained language models for text generation: A survey,” ACM Computing Surveys, vol. 56, no. 9, pp. 1–39, 2024.
  3. W. Zhu, H. Liu, Q. Dong, J. Xu, S. Huang, L. Kong, J. Chen, and L. Li, “Multilingual machine translation with large language models: Empirical results and analysis,” arXiv preprint arXiv:2304.04675, 2023.
  4. L. Wang, C. Lyu, T. Ji, Z. Zhang, D. Yu, S. Shi, and Z. Tu, “Document-level machine translation with large language models,” arXiv preprint arXiv:2304.02210, 2023.
  5. L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing et al., “Judging llm-as-a-judge with mt-bench and chatbot arena,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  6. J. K. Kim, M. Chua, M. Rickard, and A. Lorenzo, “Chatgpt and large language model (llm) chatbots: The current state of acceptability and a proposal for guidelines on utilization in academic medicine,” Journal of Pediatric Urology, vol. 19, no. 5, pp. 598–604, 2023.
  7. Y. Dan, Z. Lei, Y. Gu, Y. Li, J. Yin, J. Lin, L. Ye, Z. Tie, Y. Zhou, Y. Wang et al., “Educhat: A large-scale language model-based chatbot system for intelligent education,” arXiv preprint arXiv:2308.02773, 2023.
  8. H. Zhang, J. Xu, and J. Wang, “Pretraining-based natural language generation for text summarization,” arXiv preprint arXiv:1902.09243, 2019.
  9. B. Zhang, H. Yang, T. Zhou, M. Ali Babar, and X.-Y. Liu, “Enhancing financial sentiment analysis via retrieval augmented large language models,” in Proceedings of the fourth ACM international conference on AI in finance, 2023, pp. 349–356.
  10. Z. Pan, H. Luo, M. Li, and H. Liu, “Conv-coa: Improving open-domain question answering in large language models via conversational chain-of-action,” arXiv preprint arXiv:2405.17822, 2024.
  11. Y. Yao, J. Duan, K. Xu, Y. Cai, Z. Sun, and Y. Zhang, “A survey on large language model (llm) security and privacy: The good, the bad, and the ugly,” High-Confidence Computing, p. 100211, 2024.
  12. K. Huang, F. Mo, H. Li, Y. Li, Y. Zhang, W. Yi, Y. Mao, J. Liu, Y. Xu, J. Xu et al., “A survey on large language models with multilingualism: Recent advances and new frontiers,” arXiv preprint arXiv:2405.10936, 2024.
  13. J. Huang and K. C.-C. Chang, “Towards reasoning in large language models: A survey,” arXiv preprint arXiv:2212.10403, 2022.
  14. A. Saparov and H. He, “Language models are greedy reasoners: A systematic formal analysis of chain-of-thought,” arXiv preprint arXiv:2210.01240, 2022.
  15. N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International Conference on Machine Learning.   PMLR, 2019, pp. 2790–2799.
  16. X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021, pp. 4582–4597.
  17. B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 3045–3059.
  18. S. Chen, Y. Hou, Y. Cui, W. Che, T. Liu, and X. Yu, “Recall and learn: Fine-tuning deep pretrained language models with less forgetting,” arXiv preprint arXiv:2004.12651, 2020.
  19. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021.
  20. Q. Zhang, M. Chen, A. Bukharin, P. He, Y. Cheng, W. Chen, and T. Zhao, “Adaptive budget allocation for parameter-efficient fine-tuning,” in The Eleventh International Conference on Learning Representations, 2022.
  21. X. Liu, K. Ji, Y. Fu, W. Tam, Z. Du, Z. Yang, and J. Tang, “P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), S. Muresan, P. Nakov, and A. Villavicencio, Eds.   Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 61–68. [Online]. Available: https://aclanthology.org/2022.acl-short.8
  22. E. B. Zaken, S. Ravfogel, and Y. Goldberg, “Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models,” arXiv preprint arXiv:2106.10199, 2021.
  23. N. Ding, Y. Qin, G. Yang, F. Wei, Z. Yang, Y. Su, S. Hu, Y. Chen, C.-M. Chan, W. Chen et al., “Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models,” arXiv preprint arXiv:2203.06904, 2022.
  24. V. Lialin, V. Deshpande, and A. Rumshisky, “Scaling down to scale up: A guide to parameter-efficient fine-tuning,” arXiv preprint arXiv:2303.15647, 2023.
  25. L. Xu, H. Xie, S.-Z. J. Qin, X. Tao, and F. L. Wang, “Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment,” arXiv preprint arXiv:2312.12148, 2023.
  26. Y. Xin, S. Luo, H. Zhou, J. Du, X. Liu, Y. Fan, Q. Li, and Y. Du, “Parameter-efficient fine-tuning for pre-trained vision models: A survey,” arXiv preprint arXiv:2402.02242, 2024.
  27. Z. Han, C. Gao, J. Liu, S. Q. Zhang et al., “Parameter-efficient fine-tuning for large models: A comprehensive survey,” arXiv preprint arXiv:2403.14608, 2024.
  28. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  29. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
  30. J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al., “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023.
  31. J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler et al., “Emergent abilities of large language models,” arXiv preprint arXiv:2206.07682, 2022.
  32. V. Sanh, A. Webson, C. Raffel, S. H. Bach, L. Sutawika, Z. Alyafeai, A. Chaffin, A. Stiegler, T. L. Scao, A. Raja et al., “Multitask prompted training enables zero-shot task generalization,” arXiv preprint arXiv:2110.08207, 2021.
  33. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020.
  34. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in neural information processing systems, vol. 35, pp. 27 730–27 744, 2022.
  35. H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, M. Dehghani, S. Brahma et al., “Scaling instruction-finetuned language models,” Journal of Machine Learning Research, vol. 25, no. 70, pp. 1–53, 2024.
  36. J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” arXiv preprint arXiv:2001.08361, 2020.
  37. J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark et al., “Training compute-optimal large language models,” arXiv preprint arXiv:2203.15556, 2022.
  38. A. Radford, K. Narasimhan, T. Salimans, I. Sutskever et al., “Improving language understanding by generative pre-training,” 2018.
  39. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019.
  40. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman et al., “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374, 2021.
  41. R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders et al., “Webgpt: Browser-assisted question-answering with human feedback,” arXiv preprint arXiv:2112.09332, 2021.
  42. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” Advances in neural information processing systems, vol. 35, pp. 22 199–22 213, 2022.
  43. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023.
  44. A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier et al., “Mistral 7b,” arXiv preprint arXiv:2310.06825, 2023.
  45. C. Riquelme, J. Puigcerver, B. Mustafa, M. Neumann, R. Jenatton, A. Susano Pinto, D. Keysers, and N. Houlsby, “Scaling vision with sparse mixture of experts,” Advances in Neural Information Processing Systems, vol. 34, pp. 8583–8595, 2021.
  46. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., “Palm: Scaling language modeling with pathways,” Journal of Machine Learning Research, vol. 24, no. 240, pp. 1–113, 2023.
  47. Y. Tay, J. Wei, H. W. Chung, V. Q. Tran, D. R. So, S. Shakeri, X. Garcia, H. S. Zheng, J. Rao, A. Chowdhery et al., “Transcending scaling laws with 0.1% extra compute,” arXiv preprint arXiv:2210.11399, 2022.
  48. A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning.   PMLR, 2021, pp. 8748–8763.
  49. M. Cherti, R. Beaumont, R. Wightman, M. Wortsman, G. Ilharco, C. Gordon, C. Schuhmann, L. Schmidt, and J. Jitsev, “Reproducible scaling laws for contrastive language-image learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2818–2829.
  50. Q. Sun, Y. Fang, L. Wu, X. Wang, and Y. Cao, “Eva-clip: Improved training techniques for clip at scale,” arXiv preprint arXiv:2303.15389, 2023.
  51. P. Wang, A. Yang, R. Men, J. Lin, S. Bai, Z. Li, J. Ma, C. Zhou, J. Zhou, and H. Yang, “Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework,” in International Conference on Machine Learning.   PMLR, 2022, pp. 23 318–23 340.
  52. J. Cho, J. Lei, H. Tan, and M. Bansal, “Unifying vision-and-language tasks via text generation,” in International Conference on Machine Learning.   PMLR, 2021, pp. 1931–1942.
  53. X. Zhai, A. Kolesnikov, N. Houlsby, and L. Beyer, “Scaling vision transformers,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 12 104–12 113.
  54. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020.
  55. X. Liu, P. He, W. Chen, and J. Gao, “Multi-task deep neural networks for natural language understanding,” arXiv preprint arXiv:1901.11504, 2019.
  56. A. Aghajanyan, A. Gupta, A. Shrivastava, X. Chen, L. Zettlemoyer, and S. Gupta, “Muppet: Massive multi-task representations with pre-finetuning,” arXiv preprint arXiv:2101.11038, 2021.
  57. V. Aribandi, Y. Tay, T. Schuster, J. Rao, H. S. Zheng, S. V. Mehta, H. Zhuang, V. Q. Tran, D. Bahri, J. Ni et al., “Ext5: Towards extreme multi-task scaling for transfer learning,” arXiv preprint arXiv:2111.10952, 2021.
  58. D. Khashabi, S. Min, T. Khot, A. Sabharwal, O. Tafjord, P. Clark, and H. Hajishirzi, “Unifiedqa: Crossing format boundaries with a single qa system,” arXiv preprint arXiv:2005.00700, 2020.
  59. B. McCann, N. S. Keskar, C. Xiong, and R. Socher, “The natural language decathlon: Multitask learning as question answering. arxiv 2018,” arXiv preprint arXiv:1806.08730.
  60. N. S. Keskar, B. McCann, C. Xiong, and R. Socher, “Unifying question answering, text classification, and regression via span extraction,” arXiv preprint arXiv:1904.09286, 2019.
  61. S. Mishra, D. Khashabi, C. Baral, and H. Hajishirzi, “Cross-task generalization via natural language crowdsourcing instructions,” arXiv preprint arXiv:2104.08773, 2021.
  62. J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le, “Finetuned language models are zero-shot learners,” arXiv preprint arXiv:2109.01652, 2021.
  63. S. H. Bach, V. Sanh, Z.-X. Yong, A. Webson, C. Raffel, N. V. Nayak, A. Sharma, T. Kim, M. S. Bari, T. Fevry et al., “Promptsource: An integrated development environment and repository for natural language prompts,” arXiv preprint arXiv:2202.01279, 2022.
  64. S. Min, M. Lewis, L. Zettlemoyer, and H. Hajishirzi, “Metaicl: Learning to learn in context,” arXiv preprint arXiv:2110.15943, 2021.
  65. Y. Wang, S. Mishra, P. Alipoormolabashi, Y. Kordi, A. Mirzaei, A. Arunkumar, A. Ashok, A. S. Dhanasekaran, A. Naik, D. Stap et al., “Benchmarking generalization via in-context instructions on 1,600+ language tasks,” arXiv preprint arXiv:2204.07705, vol. 2, 2022.
  66. S. Iyer, X. V. Lin, R. Pasunuru, T. Mihaylov, D. Simig, P. Yu, K. Shuster, T. Wang, Q. Liu, P. S. Koura et al., “Opt-iml: Scaling language model instruction meta learning through the lens of generalization,” arXiv preprint arXiv:2212.12017, 2022.
  67. N. Muennighoff, T. Wang, L. Sutawika, A. Roberts, S. Biderman, T. L. Scao, M. S. Bari, S. Shen, Z.-X. Yong, H. Schoelkopf et al., “Crosslingual generalization through multitask finetuning,” arXiv preprint arXiv:2211.01786, 2022.
  68. H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, M. Dehghani, S. Brahma et al., “Scaling instruction-finetuned language models,” arXiv preprint arXiv:2210.11416, 2022.
  69. Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi, “Self-instruct: Aligning language models with self-generated instructions,” arXiv preprint arXiv:2212.10560, 2022.
  70. O. Honovich, T. Scialom, O. Levy, and T. Schick, “Unnatural instructions: Tuning language models with (almost) no human labor,” arXiv preprint arXiv:2212.09689, 2022.
  71. S. Ye, D. Kim, J. Jang, J. Shin, and M. Seo, “Guess the instruction! making language models stronger zero-shot learners,” arXiv preprint arXiv:2210.02969, 2022.
  72. P. Gupta, C. Jiao, Y.-T. Yeh, S. Mehri, M. Eskenazi, and J. P. Bigham, “Instructdial: Improving zero and few-shot generalization in dialogue through instruction tuning,” arXiv preprint arXiv:2205.12673, 2022.
  73. A. Glaese, N. McAleese, M. Trbacz, J. Aslanides, V. Firoiu, T. Ewalds, M. Rauh, L. Weidinger, M. Chadwick, P. Thacker et al., “Improving alignment of dialogue agents via targeted human judgements,” arXiv preprint arXiv:2209.14375, 2022.
  74. Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan et al., “Training a helpful and harmless assistant with reinforcement learning from human feedback,” arXiv preprint arXiv:2204.05862, 2022.
  75. Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon et al., “Constitutional ai: Harmlessness from ai feedback,” arXiv preprint arXiv:2212.08073, 2022.
  76. H. Liu, D. Tam, M. Muqeeth, J. Mohta, T. Huang, M. Bansal, and C. A. Raffel, “Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 1950–1965, 2022.
  77. T. Vu, B. Lester, N. Constant, R. Al-Rfou, and D. Cer, “Spot: Better frozen model adaptation through soft prompt transfer,” arXiv preprint arXiv:2110.07904, 2021.
  78. K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl et al., “Large language models encode clinical knowledge,” Nature, vol. 620, no. 7972, pp. 172–180, 2023.
  79. W. B. Knox and P. Stone, “Tamer: Training an agent manually via evaluative reinforcement,” in 2008 7th IEEE international conference on development and learning.   IEEE, 2008, pp. 292–297.
  80. P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Advances in neural information processing systems, vol. 30, 2017.
  81. H. Lee, S. Phatale, H. Mansoor, K. Lu, T. Mesnard, C. Bishop, V. Carbune, and A. Rastogi, “Rlaif: Scaling reinforcement learning from human feedback with ai feedback,” arXiv preprint arXiv:2309.00267, 2023.
  82. R. S. Sutton, “Generalization in reinforcement learning: Successful examples using sparse coarse coding,” in Advances in Neural Information Processing Systems, D. Touretzky, M. Mozer, and M. Hasselmo, Eds., vol. 8.   MIT Press, 1995. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/1995/file/8f1d43620bc6bb580df6e80b0dc05c48-Paper.pdf
  83. J. Fan, Z. Wang, Y. Xie, and Z. Yang, “A theoretical analysis of deep q-learning,” in Learning for dynamics and control.   PMLR, 2020, pp. 486–489.
  84. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
  85. “Common crawl,” [Online]. Available: https:// commoncrawl.org/.
  86. S. Yuan, H. Zhao, Z. Du, M. Ding, X. Liu, Y. Cen, X. Zou, Z. Yang, and J. Tang, “Wudaocorpora: A super large-scale chinese corpora for pre-training language models,” AI Open, vol. 2, pp. 65–68, 2021.
  87. J. Bandy and N. Vincent, “Addressing” documentation debt” in machine learning: A retrospective datasheet for bookcorpus,” in Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021.
  88. J. W. Rae, A. Potapenko, S. M. Jayakumar, and T. P. Lillicrap, “Compressive transformers for long-range sequence modelling,” arXiv preprint arXiv:1911.05507, 2019.
  89. “Bigquery dataset,” [Online]. Available: https:// cloud.google.com/bigquery?hl=zh-cn.
  90. D. Kocetkov, R. Li, L. B. Allal, J. Li, C. Mou, C. M. Ferrandis, Y. Jernite, M. Mitchell, S. Hughes, T. Wolf et al., “The stack: 3 tb of permissively licensed source code,” arXiv preprint arXiv:2211.15533, 2022.
  91. A. Gokaslan and V. Cohen, “Openwebtext corpus,” http://Skylion007.github.io/OpenWebTextCorpus, 2019.
  92. J. Baumgartner, S. Zannettou, B. Keegan, M. Squire, and J. Blackburn, “The pushshift reddit dataset,” in Proceedings of the international AAAI conference on web and social media, vol. 14, 2020, pp. 830–839.
  93. “Wikipedia. [online]. available: https://en. wikipedia.org/wiki/main page.”
  94. L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima et al., “The pile: An 800gb dataset of diverse text for language modeling,” arXiv preprint arXiv:2101.00027, 2020.
  95. K. Lo, L. L. Wang, M. Neumann, R. Kinney, and D. S. Weld, “S2orc: The semantic scholar open research corpus,” arXiv preprint arXiv:1911.02782, 2019.
  96. A. Eisele and Y. Chen, “Multiun: A multilingual corpus from united nation documents.” in LREC, 2010.
  97. A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “Glue: A multi-task benchmark and analysis platform for natural language understanding,” arXiv preprint arXiv:1804.07461, 2018.
  98. P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superglue: Learning feature matching with graph neural networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 4938–4947.
  99. O. Dušek, J. Novikova, and V. Rieser, “Evaluating the state-of-the-art of end-to-end natural language generation: The e2e nlg challenge,” Computer Speech & Language, vol. 59, pp. 123–156, 2020.
  100. V. Zhong, C. Xiong, and R. Socher, “Seq2sql: Generating structured queries from natural language using reinforcement learning,” CoRR, vol. abs/1709.00103, 2017.
  101. C. Gardent, A. Shimorina, S. Narayan, and L. Perez-Beltrachini, “Creating training corpora for NLG micro-planners,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, R. Barzilay and M. Kan, Eds.   Association for Computational Linguistics, 2017, pp. 179–188. [Online]. Available: https://doi.org/10.18653/v1/P17-1017
  102. B. Gliwa, I. Mochol, M. Biesek, and A. Wawer, “SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization,” in Proceedings of the 2nd Workshop on New Frontiers in Summarization.   Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 70–79. [Online]. Available: https://www.aclweb.org/anthology/D19-5409
  103. G. Wang, S. Cheng, X. Zhan, X. Li, S. Song, and Y. Liu, “Openchat: Advancing open-source language models with mixed-quality data,” arXiv preprint arXiv:2309.11235, 2023.
  104. W. Foundation. Acl 2019 fourth conference on machine translation (wmt19), shared task: Machine translation of news. [Online]. Available: http://www.statmt.org/wmt19/translation-task.html
  105. S. Narayan, S. B. Cohen, and M. Lapata, “Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization,” ArXiv, vol. abs/1808.08745, 2018.
  106. D. Radev, R. Zhang, A. Rau, A. Sivaprasad, C. Hsieh, N. F. Rajani, X. Tang, A. Vyas, N. Verma, P. Krishna, Y. Liu, N. Irwanto, J. Pan, F. Rahman, A. Zaidi, M. Mutuma, Y. Tarabar, A. Gupta, T. Yu, Y. C. Tan, X. V. Lin, C. Xiong, and R. Socher, “Dart: Open-domain structured data record to text generation,” arXiv preprint arXiv:2007.02871, 2020.
  107. J. Ji, M. Liu, J. Dai, X. Pan, C. Zhang, C. Bian, B. Chen, R. Sun, Y. Wang, and Y. Yang, “Beavertails: Towards improved safety alignment of llm via a human-preference dataset,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  108. Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning, “Hotpotqa: A dataset for diverse, explainable multi-hop question answering,” arXiv preprint arXiv:1809.09600, 2018.
  109. K. Ethayarajh, Y. Choi, and S. Swayamdipta, “Understanding dataset difficulty with 𝒱𝒱\mathcal{V}caligraphic_V-usable information,” in Proceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, Eds., vol. 162.   PMLR, 17–23 Jul 2022, pp. 5988–6008.
  110. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” Proceedings of the International Conference on Learning Representations (ICLR), 2021.
  111. P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try arc, the ai2 reasoning challenge,” arXiv:1803.05457v1, 2018.
  112. Z. Lin, A. Madotto, and P. Fung, “Exploring versatile generative language model via parameter-efficient transfer learning,” arXiv preprint arXiv:2004.03829, 2020.
  113. A. Rücklé, G. Geigle, M. Glockner, T. Beck, J. Pfeiffer, N. Reimers, and I. Gurevych, “Adapterdrop: On the efficiency of adapters in transformers,” arXiv preprint arXiv:2010.11918, 2020.
  114. H. Zhao, H. Tan, and H. Mei, “Tiny-attention adapter: Contexts are more important than the number of parameters,” arXiv preprint arXiv:2211.01979, 2022.
  115. J. He, C. Zhou, X. Ma, T. Berg-Kirkpatrick, and G. Neubig, “Towards a unified view of parameter-efficient transfer learning,” arXiv preprint arXiv:2110.04366, 2021.
  116. Y. Zhu, J. Feng, C. Zhao, M. Wang, and L. Li, “Counter-interference adapter for multilingual machine translation,” arXiv preprint arXiv:2104.08154, 2021.
  117. T. Lei, J. Bai, S. Brahma, J. Ainslie, K. Lee, Y. Zhou, N. Du, V. Zhao, Y. Wu, B. Li et al., “Conditional adapters: Parameter-efficient transfer learning with fast inference,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  118. Y. Chen, Q. Fu, G. Fan, L. Du, J.-G. Lou, S. Han, D. Zhang, Z. Li, and Y. Xiao, “Hadamard adapter: An extreme parameter-efficient adapter tuning method for pre-trained language models,” in Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023, pp. 276–285.
  119. R. Karimi Mahabadi, J. Henderson, and S. Ruder, “Compacter: Efficient low-rank hypercomplex adapter layers,” Advances in Neural Information Processing Systems, vol. 34, pp. 1022–1035, 2021.
  120. S. He, L. Ding, D. Dong, M. Zhang, and D. Tao, “Sparseadapter: An easy approach for improving the parameter-efficiency of adapters,” arXiv preprint arXiv:2210.04284, 2022.
  121. X. Liu, Y. Zheng, Z. Du, M. Ding, Y. Qian, Z. Yang, and J. Tang, “Gpt understands, too,” arXiv preprint arXiv:2103.10385, 2021.
  122. X. Liu, K. Ji, Y. Fu, W. L. Tam, Z. Du, Z. Yang, and J. Tang, “P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,” arXiv preprint arXiv:2110.07602, 2021.
  123. J.-Y. Choi, J. Kim, J.-H. Park, W.-L. Mok, and S. Lee, “Smop: Towards efficient and effective prompt tuning with sparse mixture-of-prompts,” in The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
  124. Z.-R. Zhang, C. Tan, H. Xu, C. Wang, J. Huang, and S. Huang, “Towards adaptive prefix tuning for parameter-efficient language model fine-tuning,” arXiv preprint arXiv:2305.15212, 2023.
  125. Z. Wu, S. Wang, J. Gu, R. Hou, Y. Dong, V. Vydiswaran, and H. Ma, “Idpg: An instance-dependent prompt generation method,” arXiv preprint arXiv:2204.04497, 2022.
  126. X. Liu, T. Sun, X. Huang, and X. Qiu, “Late prompt tuning: A late prompt could be better than many prompts,” arXiv preprint arXiv:2210.11292, 2022.
  127. W. Zhu and M. Tan, “Spt: Learning to selectively insert prompts for better prompt tuning,” in The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
  128. Q. Wang, Y. Mao, J. Wang, H. Yu, S. Nie, S. Wang, F. Feng, L. Huang, X. Quan, Z. Xu et al., “Aprompt: Attention prompt tuning for efficient adaptation of pre-trained language models,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 9147–9160.
  129. Z. Shi and A. Lipani, “Dept: Decomposed prompt tuning for parameter-efficient fine-tuning,” arXiv preprint arXiv:2309.05173, 2023.
  130. J. Wu, T. Yu, R. Wang, Z. Song, R. Zhang, H. Zhao, C. Lu, S. Li, and R. Henao, “Infoprompt: Information-theoretic soft prompt tuning for natural language understanding,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  131. F. Ma, C. Zhang, L. Ren, J. Wang, Q. Wang, W. Wu, X. Quan, and D. Song, “Xprompt: Exploring the extreme of prompt tuning,” arXiv preprint arXiv:2210.04457, 2022.
  132. L. Chen, H. Huang, and M. Cheng, “Ptp: Boosting stability and performance of prompt tuning with perturbation-based regularizer,” arXiv preprint arXiv:2305.02423, 2023.
  133. T. Zadouri, A. Üstün, A. Ahmadian, B. Ermiş, A. Locatelli, and S. Hooker, “Pushing mixture of experts to the limit: Extremely parameter efficient moe for instruction tuning,” arXiv preprint arXiv:2309.05444, 2023.
  134. D. Lian, D. Zhou, J. Feng, and X. Wang, “Scaling & shifting your features: A new baseline for efficient model tuning,” Advances in Neural Information Processing Systems, vol. 35, pp. 109–123, 2022.
  135. X. Yang, J. Y. Huang, W. Zhou, and M. Chen, “Parameter-efficient tuning with special token adaptation,” in Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 2023, pp. 865–872.
  136. X. Lu, F. Brahman, P. West, J. Jung, K. Chandu, A. Ravichander, P. Ammanabrolu, L. Jiang, S. Ramnath, N. Dziri et al., “Inference-time policy adapters (ipa): Tailoring extreme-scale lms without fine-tuning,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 6863–6883.
  137. Y.-L. Sung, J. Cho, and M. Bansal, “Lst: Ladder side-tuning for parameter and memory efficient transfer learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 12 991–13 005, 2022.
  138. J. Cao, C. S. Prakash, and W. Hamza, “Attention fusion: a light yet efficient late fusion mechanism for task adaptation in nlu,” in Findings of the Association for Computational Linguistics: NAACL 2022, 2022, pp. 857–866.
  139. A. Aghajanyan, L. Zettlemoyer, and S. Gupta, “Intrinsic dimensionality explains the effectiveness of language model fine-tuning,” arXiv preprint arXiv:2012.13255, 2020.
  140. A. Edalati, M. Tahaei, I. Kobyzev, V. P. Nia, J. J. Clark, and M. Rezagholizadeh, “Krona: Parameter efficient tuning with kronecker adapter,” arXiv preprint arXiv:2212.10650, 2022.
  141. M. Valipour, M. Rezagholizadeh, I. Kobyzev, and A. Ghodsi, “Dylora: Parameter-efficient tuning of pre-trained models using dynamic search-free low-rank adaptation,” in Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 2023, pp. 3274–3287.
  142. F. Zhang, L. Li, J. Chen, Z. Jiang, B. Wang, and Y. Qian, “Increlora: Incremental parameter allocation method for parameter-efficient fine-tuning,” arXiv preprint arXiv:2308.12043, 2023.
  143. N. Ding, X. Lv, Q. Wang, Y. Chen, B. Zhou, Z. Liu, and M. Sun, “Sparse low-rank adaptation of pre-trained language models,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 4133–4145.
  144. S. Hayou, N. Ghosh, and B. Yu, “Lora+: Efficient low rank adaptation of large models,” arXiv preprint arXiv:2402.12354, 2024.
  145. L. Zhang, L. Zhang, S. Shi, X. Chu, and B. Li, “Lora-fa: Memory-efficient low-rank adaptation for large language models fine-tuning,” arXiv preprint arXiv:2308.03303, 2023.
  146. S.-Y. Liu, C.-Y. Wang, H. Yin, P. Molchanov, Y.-C. F. Wang, K.-T. Cheng, and M.-H. Chen, “Dora: Weight-decomposed low-rank adaptation,” arXiv preprint arXiv:2402.09353, 2024.
  147. A. X. Yang, M. Robeyns, X. Wang, and L. Aitchison, “Bayesian low-rank adaptation for large language models,” in The Twelfth International Conference on Learning Representations, 2023.
  148. Y. Chen, D. Hazarika, M. Namazifar, Y. Liu, D. Jin, and D. Hakkani-Tur, “Empowering parameter-efficient transfer learning by recognizing the kernel structure in self-attention,” in Findings of the Association for Computational Linguistics: NAACL 2022, 2022, pp. 1375–1388.
  149. X. Meng, D. Dai, W. Luo, Z. Yang, S. Wu, X. Wang, P. Wang, Q. Dong, L. Chen, and Z. Sui, “Periodiclora: Breaking the low-rank bottleneck in lora optimization,” arXiv preprint arXiv:2402.16141, 2024.
  150. C. Tian, Z. Shi, Z. Guo, L. Li, and C. Xu, “Hydralora: An asymmetric lora architecture for efficient fine-tuning,” arXiv preprint arXiv:2404.19245, 2024.
  151. Z. Liu, S. Kundu, A. Li, J. Wan, L. Jiang, and P. A. Beerel, “Aflora: Adaptive freezing of low rank adaptation in parameter efficient fine-tuning of large models,” arXiv preprint arXiv:2403.13269, 2024.
  152. Y. Wu, Y. Xiang, S. Huo, Y. Gong, and P. Liang, “Lora-sp: Streamlined partial parameter adaptation for resource-efficient fine-tuning of large language models,” arXiv preprint arXiv:2403.08822, 2024.
  153. X. Chen, J. Liu, Y. Wang, M. Brand, G. Wang, T. Koike-Akino et al., “Superlora: Parameter-efficient unified adaptation of multi-layer attention modules,” arXiv preprint arXiv:2403.11887, 2024.
  154. D. Guo, A. M. Rush, and Y. Kim, “Parameter-efficient transfer learning with diff pruning,” arXiv preprint arXiv:2012.07463, 2020.
  155. N. Lawton, A. Kumar, G. Thattai, A. Galstyan, and G. V. Steeg, “Neural architecture search for parameter-efficient fine-tuning of large pre-trained language models,” arXiv preprint arXiv:2305.16597, 2023.
  156. B. Liao, Y. Meng, and C. Monz, “Parameter-efficient fine-tuning without introducing new latency,” arXiv preprint arXiv:2305.16742, 2023.
  157. Y.-L. Sung, V. Nair, and C. A. Raffel, “Training neural networks with fixed sparse masks,” Advances in Neural Information Processing Systems, vol. 34, pp. 24 193–24 205, 2021.
  158. S. S. S. Das, R. H. Zhang, P. Shi, W. Yin, and R. Zhang, “Unified low-resource sequence labeling by sample-aware dynamic sparse finetuning,” arXiv preprint arXiv:2311.03748, 2023.
  159. A. Ansell, E. M. Ponti, A. Korhonen, and I. Vulić, “Composable sparse fine-tuning for cross-lingual transfer,” arXiv preprint arXiv:2110.07560, 2021.
  160. Z. Fu, H. Yang, A. M.-C. So, W. Lam, L. Bing, and N. Collier, “On the effectiveness of parameter-efficient fine-tuning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 11, 2023, pp. 12 799–12 807.
  161. R. Xu, F. Luo, Z. Zhang, C. Tan, B. Chang, S. Huang, and F. Huang, “Raise a child in large language model: Towards effective and generalizable fine-tuning,” arXiv preprint arXiv:2109.05687, 2021.
  162. M. Zhao, T. Lin, F. Mi, M. Jaggi, and H. Schütze, “Masking as an efficient alternative to finetuning for pretrained language models,” arXiv preprint arXiv:2004.12406, 2020.
  163. M. Zhang, C. Shen, Z. Yang, L. Ou, X. Yu, B. Zhuang et al., “Pruning meets low-rank parameter-efficient fine-tuning,” arXiv preprint arXiv:2305.18403, 2023.
  164. D. Vucetic, M. Tayaranian, M. Ziaeefard, J. J. Clark, B. H. Meyer, and W. J. Gross, “Efficient fine-tuning of bert models on the edge,” in 2022 IEEE International Symposium on Circuits and Systems (ISCAS).   IEEE, 2022, pp. 1838–1842.
  165. M. Gheini, X. Ren, and J. May, “Cross-attention is all you need: Adapting pretrained transformers for machine translation,” arXiv preprint arXiv:2104.08771, 2021.
  166. H. He, J. Cai, J. Zhang, D. Tao, and B. Zhuang, “Sensitivity-aware visual parameter-efficient fine-tuning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 11 825–11 835.
  167. Y. Mao, L. Mathias, R. Hou, A. Almahairi, H. Ma, J. Han, W.-t. Yih, and M. Khabsa, “Unipelt: A unified framework for parameter-efficient language model tuning,” arXiv preprint arXiv:2110.07577, 2021.
  168. J. Chen, A. Zhang, X. Shi, M. Li, A. Smola, and D. Yang, “Parameter-efficient fine-tuning design spaces,” arXiv preprint arXiv:2301.01821, 2023.
  169. Z. Hu, L. Wang, Y. Lan, W. Xu, E.-P. Lim, L. Bing, X. Xu, S. Poria, and R. K.-W. Lee, “Llm-adapters: An adapter family for parameter-efficient fine-tuning of large language models,” arXiv preprint arXiv:2304.01933, 2023.
  170. Y. Zhang, K. Zhou, and Z. Liu, “Neural prompt search,” arXiv preprint arXiv:2206.04673, 2022.
  171. H. Zhou, X. Wan, I. Vulić, and A. Korhonen, “Autopeft: Automatic configuration search for parameter-efficient fine-tuning,” Transactions of the Association for Computational Linguistics, vol. 12, pp. 525–542, 2024.
  172. S. Hu, Z. Zhang, N. Ding, Y. Wang, Y. Wang, Z. Liu, and M. Sun, “Sparse structure search for delta tuning,” Advances in Neural Information Processing Systems, vol. 35, pp. 9853–9865, 2022.
  173. G. Zeng, P. Zhang, and W. Lu, “One network, many masks: Towards more parameter-efficient transfer learning,” arXiv preprint arXiv:2305.17682, 2023.
  174. S. Jie, H. Wang, and Z.-H. Deng, “Revisiting the parameter efficiency of adapters from the perspective of precision redundancy,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 17 217–17 226.
  175. J. Kim, J. H. Lee, S. Kim, J. Park, K. M. Yoo, S. J. Kwon, and D. Lee, “Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  176. T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “Qlora: Efficient finetuning of quantized llms,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  177. H. Guo, P. Greengard, E. P. Xing, and Y. Kim, “Lq-lora: Low-rank plus quantized matrix decomposition for efficient language model finetuning,” arXiv preprint arXiv:2311.12023, 2023.
  178. Y. Xu, L. Xie, X. Gu, X. Chen, H. Chang, H. Zhang, Z. Chen, X. ZHANG, and Q. Tian, “Qa-lora: Quantization-aware low-rank adaptation of large language models,” in The Twelfth International Conference on Learning Representations, 2023.
  179. H. Rajabzadeh, M. Valipour, T. Zhu, M. Tahaei, H. J. Kwon, A. Ghodsi, B. Chen, and M. Rezagholizadeh, “Qdylora: Quantized dynamic low-rank adaptation for efficient large language model tuning,” arXiv preprint arXiv:2402.10462, 2024.
  180. Y. Li, Y. Yu, C. Liang, N. Karampatziakis, P. He, W. Chen, and T. Zhao, “Loftq: Lora-fine-tuning-aware quantization for large language models,” in The Twelfth International Conference on Learning Representations, 2023.
  181. J. Liu, G. Xiao, K. Li, J. D. Lee, S. Han, T. Dao, and T. Cai, “Bitdelta: Your fine-tune may only be worth one bit,” arXiv preprint arXiv:2402.10193, 2024.
  182. J. Pfeiffer, A. Kamath, A. Rücklé, K. Cho, and I. Gurevych, “Adapterfusion: Non-destructive task composition for transfer learning,” arXiv preprint arXiv:2005.00247, 2020.
  183. Y. Wang, S. Mukherjee, X. Liu, J. Gao, A. H. Awadallah, and J. Gao, “Adamix: Mixture-of-adapter for parameter-efficient tuning of large language models,” arXiv preprint arXiv:2205.12410, vol. 1, no. 2, p. 4, 2022.
  184. H. Zhao, J. Fu, and Z. He, “Prototype-based hyperadapter for sample-efficient multi-task tuning,” arXiv preprint arXiv:2310.11670, 2023.
  185. A. Chronopoulou, M. E. Peters, A. Fraser, and J. Dodge, “Adaptersoup: Weight averaging to improve generalization of pretrained language models,” arXiv preprint arXiv:2302.07027, 2023.
  186. S. He, R.-Z. Fan, L. Ding, L. Shen, T. Zhou, and D. Tao, “Mera: Merging pretrained adapters for few-shot learning,” arXiv preprint arXiv:2308.15982, 2023.
  187. R. K. Mahabadi, S. Ruder, M. Dehghani, and J. Henderson, “Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks,” arXiv preprint arXiv:2106.04489, 2021.
  188. T. Vu, B. Lester, N. Constant, R. Al-Rfou, and D. Cer, “Spot: Better frozen model adaptation through soft prompt transfer,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 5039–5059.
  189. A. Asai, M. Salehi, M. E. Peters, and H. Hajishirzi, “Attempt: Parameter-efficient multi-task tuning via attentional mixtures of soft prompts,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 6655–6672.
  190. Z. Wang, R. Panda, L. Karlinsky, R. Feris, H. Sun, and Y. Kim, “Multitask prompt tuning enables parameter-efficient transfer learning,” in The Eleventh International Conference on Learning Representations, 2022.
  191. Y. Qin, X. Wang, Y. Su, Y. Lin, N. Ding, J. Yi, W. Chen, Z. Liu, J. Li, L. Hou et al., “Exploring universal intrinsic task subspace via prompt tuning,” arXiv preprint arXiv:2110.07867, 2021.
  192. Y. Su, X. Wang, Y. Qin, C.-M. Chan, Y. Lin, H. Wang, K. Wen, Z. Liu, P. Li, J. Li et al., “On transferability of prompt tuning for natural language processing,” arXiv preprint arXiv:2111.06719, 2021.
  193. C. Huang, Q. Liu, B. Y. Lin, T. Pang, C. Du, and M. Lin, “Lorahub: Efficient cross-task generalization via dynamic lora composition,” arXiv preprint arXiv:2307.13269, 2023.
  194. Q. Liu, X. Wu, X. Zhao, Y. Zhu, D. Xu, F. Tian, and Y. Zheng, “Moelora: An moe-based parameter efficient fine-tuning method for multi-task medical applications,” arXiv preprint arXiv:2310.18339, 2023.
  195. A. Tang, L. Shen, Y. Luo, Y. Zhan, H. Hu, B. Du, Y. Chen, and D. Tao, “Parameter efficient multi-task model fusion with partial linearization,” arXiv preprint arXiv:2310.04742, 2023.
  196. A. Agiza, M. Neseem, and S. Reda, “Mtlora: A low-rank adaptation approach for efficient multi-task learning,” arXiv preprint arXiv:2403.20320, 2024.
  197. H. Bahng, A. Jahanian, S. Sankaranarayanan, and P. Isola, “Exploring visual prompts for adapting large-scale models,” arXiv preprint arXiv:2203.17274, 2022.
  198. M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.-N. Lim, “Visual prompt tuning,” in European Conference on Computer Vision.   Springer, 2022, pp. 709–727.
  199. S. Jie and Z.-H. Deng, “Convolutional bypasses are better vision transformer adapters,” arXiv preprint arXiv:2207.07039, 2022.
  200. S. Chen, C. Ge, Z. Tong, J. Wang, Y. Song, J. Wang, and P. Luo, “Adaptformer: Adapting vision transformers for scalable visual recognition,” Advances in Neural Information Processing Systems, vol. 35, pp. 16 664–16 678, 2022.
  201. Q. Huang, X. Dong, D. Chen, W. Zhang, F. Wang, G. Hua, and N. Yu, “Diversity-aware meta visual prompting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 10 878–10 887.
  202. A. Chen, Y. Yao, P.-Y. Chen, Y. Zhang, and S. Liu, “Understanding and improving visual prompting: A label-mapping perspective,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 19 133–19 143.
  203. J. Wu, X. Li, C. Wei, H. Wang, A. Yuille, Y. Zhou, and C. Xie, “Unleashing the power of visual prompting at the pixel level,” TMLR, 2024.
  204. C.-H. Tu, Z. Mai, and W.-L. Chao, “Visual query tuning: Towards effective usage of intermediate representations for parameter and memory efficient transfer learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7725–7735.
  205. S. Jie and Z.-H. Deng, “Fact: Factor-tuning for lightweight adaptation on vision transformer,” in Proceedings of the AAAI conference on artificial intelligence, vol. 37, no. 1, 2023, pp. 1060–1068.
  206. M. Fu, K. Zhu, and J. Wu, “Dtl: Disentangled transfer learning for visual recognition,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 11, 2024, pp. 12 082–12 090.
  207. H. Wang, J. Chang, Y. Zhai, X. Luo, J. Sun, Z. Lin, and Q. Tian, “Lion: Implicit vision prompt tuning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 6, 2024, pp. 5372–5380.
  208. Y.-C. Liu, C.-Y. Ma, J. Tian, Z. He, and Z. Kira, “Polyhistor: Parameter-efficient multi-task adaptation for dense vision tasks,” Advances in Neural Information Processing Systems, vol. 35, pp. 36 889–36 901, 2022.
  209. Z. Chen, Y. Duan, W. Wang, J. He, T. Lu, J. Dai, and Y. Qiao, “Vision transformer adapter for dense predictions,” arXiv preprint arXiv:2205.08534, 2022.
  210. M. Xu, Z. Zhang, F. Wei, H. Hu, and X. Bai, “Side adapter network for open-vocabulary semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2945–2954.
  211. D. Yin, Y. Yang, Z. Wang, H. Yu, K. Wei, and X. Sun, “1% vs 100%: Parameter-efficient low rank adapter for dense predictions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 20 116–20 126.
  212. N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22 500–22 510.
  213. R. Gal, Y. Alaluf, Y. Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or, “An image is worth one word: Personalizing text-to-image generation using textual inversion,” arXiv preprint arXiv:2208.01618, 2022.
  214. Z. Dong, P. Wei, and L. Lin, “Dreamartist: Towards controllable one-shot text-to-image generation via positive-negative prompt-tuning,” arXiv preprint arXiv:2211.11337, 2022.
  215. A. Voynov, Q. Chu, D. Cohen-Or, and K. Aberman, “p+limit-from𝑝p+italic_p +: Extended textual conditioning in text-to-image generation,” arXiv preprint arXiv:2303.09522, 2023.
  216. E. Xie, L. Yao, H. Shi, Z. Liu, D. Zhou, Z. Liu, J. Li, and Z. Li, “Difffit: Unlocking transferability of large diffusion models via simple parameter-efficient fine-tuning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4230–4239.
  217. Z. Liu, R. Feng, K. Zhu, Y. Zhang, K. Zheng, Y. Liu, D. Zhao, J. Zhou, and Y. Cao, “Cones: Concept neurons in diffusion models for customized generation,” arXiv preprint arXiv:2303.05125, 2023.
  218. L. Han, Y. Li, H. Zhang, P. Milanfar, D. Metaxas, and F. Yang, “Svdiff: Compact parameter space for diffusion fine-tuning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 7323–7334.
  219. S.-Y. Yeh, Y.-G. Hsieh, Z. Gao, B. B. Yang, G. Oh, and Y. Gong, “Navigating text-to-image customization: From lycoris fine-tuning to model evaluation,” arXiv preprint arXiv:2309.14859, 2023.
  220. S. Marjit, H. Singh, N. Mathur, S. Paul, C.-M. Yu, and P.-Y. Chen, “Diffusekrona: A parameter efficient fine-tuning method for personalized diffusion model,” arXiv preprint arXiv:2402.17412, 2024.
  221. Z. Qiu, W. Liu, H. Feng, Y. Xue, Y. Feng, Z. Liu, D. Zhang, A. Weller, and B. Schölkopf, “Controlling text-to-image diffusion by orthogonal finetuning,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  222. A. Voynov, K. Aberman, and D. Cohen-Or, “Sketch-guided text-to-image diffusion models,” in ACM SIGGRAPH 2023 Conference Proceedings, 2023, pp. 1–11.
  223. L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3836–3847.
  224. C. Mou, X. Wang, L. Xie, Y. Wu, J. Zhang, Z. Qi, Y. Shan, and X. Qie, “T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models,” arXiv preprint arXiv:2302.08453, 2023.
  225. S. Zhao, D. Chen, Y.-C. Chen, J. Bao, S. Hao, L. Yuan, and K.-Y. K. Wong, “Uni-controlnet: All-in-one control to text-to-image diffusion models,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  226. H. Ye, J. Zhang, S. Liu, X. Han, and W. Yang, “Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models,” arXiv preprint arXiv:2308.06721, 2023.
  227. J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” arXiv preprint arXiv:2301.12597, 2023.
  228. H. Liu, C. Li, Y. Li, and Y. J. Lee, “Improved baselines with visual instruction tuning,” arXiv preprint arXiv:2310.03744, 2023.
  229. H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, 2024.
  230. J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds et al., “Flamingo: a visual language model for few-shot learning,” Advances in neural information processing systems, vol. 35, pp. 23 716–23 736, 2022.
  231. R. Zhang, J. Han, C. Liu, P. Gao, A. Zhou, X. Hu, S. Yan, P. Lu, H. Li, and Y. Qiao, “Llama-adapter: Efficient fine-tuning of language models with zero-init attention,” arXiv preprint arXiv:2303.16199, 2023.
  232. W. Wang, Q. Lv, W. Yu, W. Hong, J. Qi, Y. Wang, J. Ji, Z. Yang, L. Zhao, X. Song et al., “Cogvlm: Visual expert for pretrained language models,” arXiv preprint arXiv:2311.03079, 2023.
  233. W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P. N. Fung, and S. Hoi, “Instructblip: Towards general-purpose vision-language models with instruction tuning,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  234. B. Yang, H. Tian, J. Ren, H. Zhang, J. Klein, T. F. Bissyandé, C. L. Goues, and S. Jin, “Multi-objective fine-tuning for enhanced program repair with llms,” arXiv preprint arXiv:2404.12636, 2024.
  235. X. Zhou, J. He, Y. Ke, G. Zhu, V. Gutiérrez-Basulto, and J. Z. Pan, “An empirical study on parameter-efficient fine-tuning for multimodal large language models,” arXiv preprint arXiv:2406.05130, 2024.
  236. Z. Jiang, J. Araki, H. Ding, and G. Neubig, “How can we know when language models know? on the calibration of language models for question answering,” Transactions of the Association for Computational Linguistics, vol. 9, pp. 962–977, 2021.
  237. K. Tian, E. Mitchell, A. Zhou, A. Sharma, R. Rafailov, H. Yao, C. Finn, and C. D. Manning, “Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback,” arXiv preprint arXiv:2305.14975, 2023.
  238. J. Lee, N. Stevens, S. C. Han, and M. Song, “A survey of large language models in finance (finllms),” arXiv preprint arXiv:2402.02315, 2024.
  239. K. Huang, Y. Qu, H. Cousins, W. A. Johnson, D. Yin, M. Shah, D. Zhou, R. Altman, M. Wang, and L. Cong, “Crispr-gpt: An llm agent for automated design of gene-editing experiments,” arXiv preprint arXiv:2404.18021, 2024.
  240. M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, 2016, pp. 308–318.
  241. X. Li, F. Tramer, P. Liang, and T. Hashimoto, “Large language models can be strong differentially private learners,” arXiv preprint arXiv:2110.05679, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Luping Wang (7 papers)
  2. Sheng Chen (133 papers)
  3. Linnan Jiang (2 papers)
  4. Shu Pan (2 papers)
  5. Runze Cai (6 papers)
  6. Sen Yang (191 papers)
  7. Fei Yang (110 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets