Sentiment trading with large language models (2412.19245v1)

Abstract: We investigate the efficacy of LLMs in sentiment analysis of U.S. financial news and their potential in predicting stock market returns. We analyze a dataset comprising 965,375 news articles that span from January 1, 2010, to June 30, 2023; we focus on the performance of various LLMs, including BERT, OPT, FINBERT, and the traditional Loughran-McDonald dictionary model, which has been a dominant methodology in the finance literature. The study documents a significant association between LLM scores and subsequent daily stock returns. Specifically, OPT, which is a GPT-3 based LLM, shows the highest accuracy in sentiment prediction with an accuracy of 74.4%, slightly ahead of BERT (72.5%) and FINBERT (72.2%). In contrast, the Loughran-McDonald dictionary model demonstrates considerably lower effectiveness with only 50.1% accuracy. Regression analyses highlight a robust positive impact of OPT model scores on next-day stock returns, with coefficients of 0.274 and 0.254 in different model specifications. BERT and FINBERT also exhibit predictive relevance, though to a lesser extent. Notably, we do not observe a significant relationship between the Loughran-McDonald dictionary model scores and stock returns, challenging the efficacy of this traditional method in the current financial context. In portfolio performance, the long-short OPT strategy excels with a Sharpe ratio of 3.05, compared to 2.11 for BERT and 2.07 for FINBERT long-short strategies. Strategies based on the Loughran-McDonald dictionary yield the lowest Sharpe ratio of 1.23. Our findings emphasize the superior performance of advanced LLMs, especially OPT, in financial market prediction and portfolio management, marking a significant shift in the landscape of financial analysis tools with implications to financial regulation and policy analysis.

• The paper shows that LLMs, particularly the GPT-3-based OPT model, achieve 74.4% prediction accuracy in financial sentiment analysis.
• It utilizes a two-step process that calibrates financial texts with LLMs followed by econometric modeling with firm- and date-specific fixed effects.
• The findings indicate that LLM-powered strategies yield a Sharpe ratio of 3.05 and a 355% return, outperforming traditional approaches.

Sentiment Trading with LLMs

The paper "Sentiment Trading with LLMs" by Kemal Kirtac and Guido Germano presents an in-depth analysis of the impact and efficacy of using advanced natural language processing techniques, specifically LLMs, for sentiment analysis in the context of financial market predictions. The research primarily evaluates the predictive power of different LLMs, including the Open Pre-trained Transformer (OPT), BERT, and its financial variant FinBERT, to assess their ability to interpret sentiment from financial news and make investment decisions based on these interpretations.

Summary of Key Findings

This paper utilizes a sizable dataset of 965,375 U.S. financial news articles spanning 2010 to 2023 to test the prediction accuracy of various models. The results demonstrate that the GPT-3-based OPT model significantly outperforms others, achieving an impressive stock market prediction accuracy of 74.4%. In terms of financial strategies, the OPT-based long-short strategy yields a noteworthy Sharpe ratio of 3.05 and a return of 355% over a period from August 2021 to July 2023, effectively outstripping other strategies and traditional portfolios. This underlines the transformative application potential of LLMs in financial prediction and portfolio management.

Methodological Approach

The paper differentiates itself by employing a robust methodological framework that leverages a two-step process, starting with the calibration of financial texts into a structured numerical format using LLMs, followed by econometric modeling to predict financial outcomes. This involves an intricate process of fine-tuning BERT and OPT models using a financial sentiment index based on three-day aggregated excess returns. These LLMs are compared against the traditional sentiment analysis through the Loughran-McDonald dictionary, commonly used in the finance domain.

In their econometric analysis, Kirtac and Germano adopt a linear regression framework embedded with firm- and date-specific fixed effects to assess the prediction power of these LLMs. The results indicate a significantly higher predictive strength for LLMs, particularly OPT, compared to the traditional dictionary method.

Implications and Future Prospects

The findings from this research offer substantial implications for investment strategy development, financial market prediction, and the broader application of AI in finance. The utilization of LLMs such as OPT and BERT in financial modeling signifies a departure from traditional sentiment analysis approaches, highlighting the potential for enhanced accuracy and performance in stock prediction.

The paper opens a multitude of avenues for future research, particularly in the development of more domain-specific LLMs that can further harness complex financial text data for predictive analytics. Additionally, it emphasizes the need for regulators and policymakers to consider the implications of increasing LLM applications in financial markets, potentially influencing market behavior, information dissemination, and price formation dynamics.

Conclusion

Kirtac and Germano's research provides compelling evidence on the efficacy of LLMs in financial sentiment analysis and stock market prediction. By demonstrating the capabilities of these advanced models, this paper lays the groundwork for integrating AI-driven decision-making tools in finance, encouraging further exploration into sophisticated LLM applications tailored to the unique demands of the financial sector. This extends the frontier of research in financial economics and contributes to a better understanding of how textual data, when effectively analyzed using AI, can serve as a powerful predictor of financial market movements.