Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models (2401.07159v1)

Published 13 Jan 2024 in cs.LG and cs.AI

Abstract: Finetuning LLMs has been empirically effective on a variety of downstream tasks. Existing approaches to finetuning an LLM either focus on parameter-efficient finetuning, which only updates a small number of trainable parameters, or attempt to reduce the memory footprint during the training phase of the finetuning. Typically, the memory footprint during finetuning stems from three contributors: model weights, optimizer states, and intermediate activations. However, existing works still require considerable memory and none can simultaneously mitigate memory footprint for all three sources. In this paper, we present Quantized Side Tuing (QST), which enables memory-efficient and fast finetuning of LLMs by operating through a dual-stage process. First, QST quantizes an LLM's model weights into 4-bit to reduce the memory footprint of the LLM's original weights; QST also introduces a side network separated from the LLM, which utilizes the hidden states of the LLM to make task-specific predictions. Using a separate side network avoids performing backpropagation through the LLM, thus reducing the memory requirement of the intermediate activations. Furthermore, QST leverages several low-rank adaptors and gradient-free downsample modules to significantly reduce the trainable parameters, so as to save the memory footprint of the optimizer states. Experiments show that QST can reduce the total memory footprint by up to 2.3 $\times$ and speed up the finetuning process by up to 3 $\times$ while achieving competent performance compared with the state-of-the-art. When it comes to full finetuning, QST can reduce the total memory footprint up to 7 $\times$.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861.
  2. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862.
  3. The fifth pascal recognizing textual entailment challenge. TAC, 7:8.
  4. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  5. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055.
  6. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174.
  7. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
  8. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314.
  9. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  10. Bill Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Third International Workshop on Paraphrasing (IWP2005).
  11. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.
  12. Krona: Parameter efficient tuning with kronecker adapter. arXiv preprint arXiv:2212.10650.
  13. Luciano Floridi and Massimo Chiriatti. 2020. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30:681–694.
  14. Jonathan Frankle and Michael Carbin. 2018. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635.
  15. Linear mode connectivity and the lottery ticket hypothesis. In International Conference on Machine Learning, pages 3259–3269. PMLR.
  16. The reversible residual network: Backpropagation without storing activations. Advances in neural information processing systems, 30.
  17. Towards a unified view of parameter-efficient transfer learning. arXiv preprint arXiv:2110.04366.
  18. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.
  19. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531.
  20. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pages 2790–2799. PMLR.
  21. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.
  22. Shankar Iyer. 2017. First quora dataset release: Question pairs. https://quoradata.quora.com/First-Quora-Dataset-Release- Question-Pairs.
  23. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  24. Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451.
  25. Lit: Learned intermediate representation training for model compression. In International Conference on Machine Learning, pages 3509–3518. PMLR.
  26. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324.
  27. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691.
  28. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190.
  29. Make your pre-trained model reversible: From parameter to memory efficient fine-tuning. arXiv preprint arXiv:2306.00477.
  30. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35:1950–1965.
  31. Gpt understands, too. AI Open.
  32. Reversible vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10830–10840.
  33. Unipelt: A unified framework for parameter-efficient language model tuning. arXiv preprint arXiv:2110.07577.
  34. Brian W Matthews. 1975. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure, 405(2):442–451.
  35. Metaicl: Learning to learn in context. arXiv preprint arXiv:2110.15943.
  36. OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774.
  37. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744.
  38. Automatic differentiation in pytorch.
  39. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247.
  40. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  41. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551.
  42. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
  43. Progressive neural networks. arXiv preprint arXiv:1606.04671.
  44. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100.
  45. Timo Schick and Hinrich Schütze. 2020. Exploiting cloze questions for few shot text classification and natural language inference. arXiv preprint arXiv:2001.07676.
  46. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631–1642.
  47. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008–3021.
  48. Lst: Ladder side-tuning for parameter and memory efficient transfer learning. Advances in Neural Information Processing Systems, 35:12991–13005.
  49. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca.
  50. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  51. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461.
  52. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560.
  53. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. arXiv preprint arXiv:2204.07705.
  54. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625–641.
  55. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  56. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426.
  57. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
  58. Side-tuning: a baseline for network adaptation via additive side networks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pages 698–714. Springer.
  59. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068.
  60. Judging llm-as-a-judge with mt-bench and chatbot arena.
  61. Autopeft: Automatic configuration search for parameter-efficient fine-tuning. arXiv preprint arXiv:2301.12132.
  62. Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now