CatMemo at the FinLLM Challenge Task: Fine-Tuning Large Language Models using Data Fusion in Financial Applications (2407.01953v1)
Abstract: The integration of LLMs into financial analysis has garnered significant attention in the NLP community. This paper presents our solution to IJCAI-2024 FinLLM challenge, investigating the capabilities of LLMs within three critical areas of financial tasks: financial classification, financial text summarization, and single stock trading. We adopted Llama3-8B and Mistral-7B as base models, fine-tuning them through Parameter Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA) approaches. To enhance model performance, we combine datasets from task 1 and task 2 for data fusion. Our approach aims to tackle these diverse tasks in a comprehensive and integrated manner, showcasing LLMs' capacity to address diverse and complex financial tasks with improved accuracy and decision-making capabilities.
- Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
- AI@Meta. 2024. Llama 3 model card.
- A survey of fintech research and policy discussion. Review of Corporate Finance, 1:259–339.
- Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712.
- Natural language processing in accounting, auditing and finance: A synthesis of the literature with a roadmap for future research. Intelligent Systems in Accounting, Finance and Management, 23(3):157–214.
- Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.
- Multi-view fusion for instruction mining of large language model. Information Fusion, page 102480.
- Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2704–2713.
- Mistral 7b. arXiv preprint arXiv:2310.06825.
- Large language models in finance: A survey. In Proceedings of the Fourth ACM International Conference on AI in Finance, pages 374–382.
- Peft: State-of-the-art parameter-efficient fine-tuning methods. https://github.com/huggingface/peft.
- An information fusion based approach to context-based fine-tuning of gpt models. Information Fusion, 104:102202.
- Fine-grained argument understanding with bert ensemble techniques: A deep dive into financial sentiment analysis. In Proceedings of the 35th Conference on Computational Linguistics and Speech Processing (ROCLING 2023), pages 242–249.
- Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
- The finben: An holistic financial benchmark for large language models. arXiv preprint arXiv:2402.12659.
- Finmem: A performance-enhanced llm trading agent with layered memory and character design. In Proceedings of the AAAI Symposium Series, volume 3, pages 595–597.
- Trade the event: Corporate events detection for news-based event-driven trading. arXiv preprint arXiv:2105.12825.
- Yupeng Cao (15 papers)
- Zhiyuan Yao (31 papers)
- Zhi Chen (235 papers)
- Zhiyang Deng (7 papers)