Are LLMs Rational Investors? A Study on Detecting and Reducing the Financial Bias in LLMs (2402.12713v2)
Abstract: LLMs are increasingly adopted in financial analysis for interpreting complex market data and trends. However, their use is challenged by intrinsic biases (e.g., risk-preference bias) and a superficial understanding of market intricacies, necessitating a thorough assessment of their financial insight. To address these issues, we introduce Financial Bias Indicators (FBI), a framework with components like Bias Unveiler, Bias Detective, Bias Tracker, and Bias Antidote to identify, detect, analyze, and eliminate irrational biases in LLMs. By combining behavioral finance principles with bias examination, we evaluate 23 leading LLMs and propose a de-biasing method based on financial causal knowledge. Results show varying degrees of financial irrationality among models, influenced by their design and training. Models trained specifically on financial datasets may exhibit more irrationality, and even larger financial LLMs (FinLLMs) can show more bias than smaller, general models. We utilize four prompt-based methods incorporating causal debiasing, effectively reducing financial biases in these models. This work enhances the understanding of LLMs' bias in financial applications, laying the foundation for developing more reliable and rational financial analysis tools.
- Qwen technical report. arXiv preprint arXiv:2309.16609.
- Nicholas Barberis and Richard Thaler. 2003. A survey of behavioral finance. Handbook of the Economics of Finance, 1:1053–1128.
- Finqa: A dataset of numerical reasoning over financial data. arXiv preprint arXiv:2109.00122.
- Buy tesla, sell ford: Assessing implicit stock market preference in pre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 100–105.
- Chatlaw: Open-source legal large language model with integrated external knowledge bases.
- Can large language models beat wall street? unveiling the potential of ai in stock selection. arXiv preprint arXiv:2401.03737.
- Bias and fairness in large language models: A survey.
- Maarten Grootendorst. 2022. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794.
- Opiniongpt: Modelling explicit biases in instruction-tuned llms.
- Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.
- Stereomap: Quantifying the awareness of human-like stereotypes in large language models. arXiv preprint arXiv:2310.13673.
- Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.
- Alejandro Lopez-Lira and Yuehua Tang. 2023. Can chatgpt forecast stock price movements? return predictability and large language models. arXiv preprint arXiv:2304.07619.
- Stereoset: Measuring stereotypical bias in pretrained language models.
- OpenAI. 2023. Gpt-4 technical report.
- Chatgpt-based investment portfolio selection. In Operations Research Forum, volume 4, page 91. Springer.
- The self-perception and political biases of chatgpt. Human Behavior and Emerging Technologies, 2024.
- Mathematics for economists, volume 7. Norton New York.
- Analysis of variance (anova). Chemometrics and intelligent laboratory systems, 6(4):259–272.
- Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
- Huatuo: Tuning llama model with chinese medical knowledge.
- Xia Wang and Min Wu. 2011. The quality of financial reporting in china: An examination from an accounting restatement perspective. China journal of accounting research, 4(4):167–196.
- Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
- Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100.
- Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564.
- Measuring consistency in text-based financial forecasting models. arXiv preprint arXiv:2305.08524.
- Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414.
- Xuanyuan 2.0: A large chinese financial chat model with hundreds of billions parameters.
- A survey of large language models.
- Large language models are not robust multiple choice selectors. arXiv e-prints, pages arXiv–2309.
- Trade the event: Corporate events detection for news-based event-driven trading. arXiv preprint arXiv:2105.12825.
- Yuhang Zhou (52 papers)
- Yuchen Ni (4 papers)
- Xiang Liu (475 papers)
- Jian Zhang (542 papers)
- Sen Liu (35 papers)
- Guangnan Ye (17 papers)
- Hongfeng Chai (7 papers)
- Yunhui Gan (1 paper)
- Zhangyue Yin (27 papers)
- Xipeng Qiu (257 papers)