Predicting Financial Markets: Comparing Survey, News, Twitter and Search Engine Data (1112.1051v1)

Abstract: Financial market prediction on the basis of online sentiment tracking has drawn a lot of attention recently. However, most results in this emerging domain rely on a unique, particular combination of data sets and sentiment tracking tools. This makes it difficult to disambiguate measurement and instrument effects from factors that are actually involved in the apparent relation between online sentiment and market values. In this paper, we survey a range of online data sets (Twitter feeds, news headlines, and volumes of Google search queries) and sentiment tracking methods (Twitter Investor Sentiment, Negative News Sentiment and Tweet & Google Search volumes of financial terms), and compare their value for financial prediction of market indices such as the Dow Jones Industrial Average, trading volumes, and market volatility (VIX), as well as gold prices. We also compare the predictive power of traditional investor sentiment survey data, i.e. Investor Intelligence and Daily Sentiment Index, against those of the mentioned set of online sentiment indicators. Our results show that traditional surveys of Investor Intelligence are lagging indicators of the financial markets. However, weekly Google Insight Search volumes on financial search queries do have predictive value. An indicator of Twitter Investor Sentiment and the frequency of occurrence of financial terms on Twitter in the previous 1-2 days are also found to be very statistically significant predictors of daily market log return. Survey sentiment indicators are however found not to be statistically significant predictors of financial market values, once we control for all other mood indicators as well as the VIX.

Predicting Financial Markets: An Analysis of Survey, News, Twitter, and Search Engine Data

The paper "Predicting Financial Markets: Comparing Survey, News, Twitter and Search Engine Data" investigates the predictive potential of various online sentiment indicators on financial market indices. The research is particularly concerned with disentangling the measurement and instrument effects inherent in previous studies and establishing a more comprehensive understanding of how public sentiment, as measured across multiple online platforms, correlates with financial outcomes.

Data Sources and Sentiment Indicators

The authors employed a multi-pronged approach to sentiment analysis, leveraging data from social media (Twitter), news media, search engines (Google Insights), and traditional survey methods to create a spectrum of sentiment indicators. These included Twitter Investor Sentiment (TIS), Negative News Sentiment (NNS), Google Search volumes (GIS), and survey-based indicators like the Daily Sentiment Index (DSI) and Investor Intelligence (II). This broad methodological base addresses a significant gap in previous research, which typically relied on a singular data type.

Key Findings

1. Predictive Power of Sentiment Indicators:
   • The paper demonstrates that online sentiment indicators, particularly Twitter-based metrics (TIS, TV-FST), show statistically significant predictive power for daily market returns. Notably, this predictive capacity surpasses traditional survey methods like DSI.
   • GIS data revealed strong correlations with financial indices, notably the DJIA, VIX, and trading volumes. The paper notably found that GIS could be indicative of market conditions, suggesting it as a computational gauge of investor sentiment or "fear."
2. Time-Lag Correlation:
   • The analysis of lagged data found that certain sentiment indicators could predict financial indices with varying time lags, highlighting potential windows for financial forecasting based on social media activity and search engine queries.
3. Granger Causality and Directional Accuracy:
   • Granger causality tests confirmed the predictive power of online sentiment for various financial indicators, refuting the hypothesis that traditional survey data/II significantly forecast financial markets.
   • The incorporation of sentiment indicators improved forecasting models, notably in times of high volatility, as observed in the August 2011 financial downturn.

Theoretical Implications

This paper contributes to the field of behavioral finance by empirically substantiating the role of online sentiment as a predictive tool for financial markets. By employing a diversified set of sentiment indicators, the research challenges the efficient market hypothesis, affirming that behavioral factors—especially those captured in real-time from online sources—carry substantial predictive information. The findings suggest an increasing shift towards digital sentiment analysis in economic forecasting, emphasizing computational methods over traditional surveys.

Practical Implications and Future Directions

From a practical perspective, this research underscores the utility of high-frequency online sentiment data in developing predictive financial models. Financial analysts and traders could employ these insights to enhance trading strategies, especially during periods of market uncertainty or volatility.

Future research should explore more nuanced machine learning models capable of capturing non-linear relationships between sentiment indicators and financial indices. Additionally, extending the analysis to other financial markets and geographic regions would provide a broader validation of these findings. Explorations into the ethical implications of using real-time public sentiment for financial gain have also become increasingly pertinent.

In conclusion, the paper significantly advances our understanding of how diverse sentiment indicators derive from varied online platforms can enhance the prediction of financial market movements, pointing towards a future where behavioral indicators are central to economic forecasting.