Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DAM: A Universal Dual Attention Mechanism for Multimodal Timeseries Cryptocurrency Trend Forecasting (2405.00522v1)

Published 1 May 2024 in econ.GN, q-fin.EC, cs.CE, cs.CL, cs.CR, and q-fin.CP

Abstract: In the distributed systems landscape, Blockchain has catalyzed the rise of cryptocurrencies, merging enhanced security and decentralization with significant investment opportunities. Despite their potential, current research on cryptocurrency trend forecasting often falls short by simplistically merging sentiment data without fully considering the nuanced interplay between financial market dynamics and external sentiment influences. This paper presents a novel Dual Attention Mechanism (DAM) for forecasting cryptocurrency trends using multimodal time-series data. Our approach, which integrates critical cryptocurrency metrics with sentiment data from news and social media analyzed through CryptoBERT, addresses the inherent volatility and prediction challenges in cryptocurrency markets. By combining elements of distributed systems, natural language processing, and financial forecasting, our method outperforms conventional models like LSTM and Transformer by up to 20\% in prediction accuracy. This advancement deepens the understanding of distributed systems and has practical implications in financial markets, benefiting stakeholders in cryptocurrency and blockchain technologies. Moreover, our enhanced forecasting approach can significantly support decentralized science (DeSci) by facilitating strategic planning and the efficient adoption of blockchain technologies, improving operational efficiency and financial risk management in the rapidly evolving digital asset domain, thus ensuring optimal resource allocation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (71)
  1. Li, Q., Tan, J., Wang, J., Chen, H. “A multimodal event-driven lstm model for stock prediction using online news.” IEEE Transactions on Knowledge and Data Engineering 33.10 3323–3337 (2020).
  2. Bhatt, S., Ghazanfar, M., Amirhosseini, M. “Machine Learning based Cryptocurrency Price Prediction using historical data and Social Media Sentiment.” Computer Science & Information Technology (CS & IT) 13.10 1–11 (2023).
  3. Sardelich, M., Manandhar, S. “Multimodal deep learning for short-term stock volatility prediction.” arXiv preprint arXiv:1812.10479 .
  4. Shin, H.-G., Ra, I., Choi, Y.-H. “A deep multimodal reinforcement learning system combined with CNN and LSTM for stock trading.” In 2019 International conference on information and communication technology convergence (ICTC) IEEE 7–11 (2019) .
  5. Zhang, X., Zhang, Y., Wang, S., Yao, Y., Fang, B., Philip, S. Y. “Improving stock market prediction via heterogeneous information fusion.” Knowledge-Based Systems 143 236–247 (2018).
  6. Anamika, Chakraborty, M., Subramaniam, S. “Does sentiment impact cryptocurrency?” Journal of Behavioral Finance 24.2 202–218 (2023).
  7. Sapkota, N. “News-based sentiment and bitcoin volatility.” International Review of Financial Analysis 82 102183 (2022).
  8. Raju, S., Tarif, A. M. “Real-time prediction of BITCOIN price using machine learning techniques and public sentiment analysis.” arXiv preprint arXiv:2006.14473 .
  9. Parekh, R., et al. “DL-GuesS: Deep learning and sentiment analysis-based cryptocurrency price prediction.” IEEE Access 10 35398–35409 (2022).
  10. Abraham, J., Higdon, D., Nelson, J., Ibarra, J. “Cryptocurrency price prediction using tweet volumes and sentiment analysis.” SMU Data Science Review 1.3 1 (2018).
  11. Kraaijeveld, O., De Smedt, J. “The predictive power of public Twitter sentiment for forecasting cryptocurrency prices.” Journal of International Financial Markets, Institutions and Money 65 101188 (2020).
  12. Rognone, L., Hyde, S., Zhang, S. S. “News sentiment in the cryptocurrency market: An empirical comparison with Forex.” International Review of Financial Analysis 69 101462 (2020).
  13. Huy, N. H., Dao, B., Mai, T.-T., Nguyen-An, K., et al. “Predicting cryptocurrency price movements based on Social Media.” In 2019 International Conference on Advanced Computing and Applications (ACOMP) IEEE 57–64 (2019) .
  14. Fu, Y., Zhuang, Z., Zhang, L. “Ai ethics on blockchain: Topic analysis on twitter data for blockchain security.” In Science and Information Conference Springer 82–100 (2023) .
  15. Antweiler, W., Frank, M. Z. “Do US stock markets typically overreact to corporate news stories?” Available at SSRN 878091 .
  16. Fu, Y., Zhou, M., Zhang, L. “Replication Data for: “DAM: A Universal Dual Attention Mechanism for Multimodal Timeseries Cryptocurrency Trend Forecasting”.” (2024) doi:10.7910/DVN/HWBT53 URL https://doi.org/10.7910/DVN/HWBT53.
  17. Herlihy, M. “Blockchains from a distributed computing perspective.” Communications of the ACM 62.2 78–85 (2019).
  18. Hashemi Joo, M., Nishikawa, Y., Dandapani, K. “Cryptocurrency, a successful application of blockchain technology.” Managerial Finance 46.6 715–733 (2020).
  19. Hu, Z., et al. “A Data Flow Framework with High Throughput and Low Latency for Permissioned Blockchains.” In 2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS) IEEE 1–12 (2023) .
  20. Shae, Z., Tsai, J. “Transform blockchain into distributed parallel computing architecture for precision medicine.” In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS) IEEE 1290–1299 (2018) .
  21. Ng, L. K., Chow, S. S., Wong, D. P., Woo, A. P. “LDSP: shopping with cryptocurrency privately and quickly under leadership.” In 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS) IEEE 261–271 (2021) .
  22. Chen, K.-Y., Lee, P.-J., Liu, S.-C. “Poster: Stock Price Prediction Using Machine Learning.” In 2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS) IEEE 1067–1068 (2023) .
  23. Almeida, J., Gonçalves, T. C. “A systematic literature review of investor behavior in the cryptocurrency markets.” Journal of Behavioral and Experimental Finance 100785.
  24. Hutto, C., Gilbert, E. “Vader: A parsimonious rule-based model for sentiment analysis of social media text.” In Proceedings of the international AAAI conference on web and social media. 8 216–225 (2014) .
  25. Gurrib, I., Kamalov, F. “Predicting bitcoin price movements using sentiment analysis: a machine learning approach.” Studies in Economics and Finance 39.3 347–364 (2022).
  26. Critien, J. V., Gatt, A., Ellul, J. “Bitcoin price change and trend prediction through twitter sentiment and data volume.” Financial Innovation 8.1 1–20 (2022).
  27. Min, B., et al. “Recent advances in natural language processing via large pre-trained language models: A survey.” ACM Computing Surveys 56.2 1–40 (2023).
  28. Passalis, N., et al. “Multisource financial sentiment analysis for detecting Bitcoin price change indications using deep learning.” Neural Computing and Applications 34.22 19441–19452 (2022).
  29. Sridhar, S., Sanagavarapu, S. “Multi-head self-attention transformer for dogecoin price prediction.” In 2021 14th International Conference on Human System Interaction (HSI) IEEE 1–6 (2021) .
  30. Muhammad, T., et al. “Transformer-based deep learning model for stock price prediction: A case study on Bangladesh stock market.” International Journal of Computational Intelligence and Applications 2350013.
  31. Yoo, J., Soun, Y., Park, Y.-c., Kang, U. “Accurate multivariate stock movement prediction via data-axis transformer with multi-level contexts.” In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining 2037–2045 (2021) .
  32. Zeng, A., Chen, M., Zhang, L., Xu, Q. “Are transformers effective for time series forecasting?” In Proceedings of the AAAI conference on artificial intelligence. 37 11121–11128 (2023) .
  33. Shahi, T. B., Shrestha, A., Neupane, A., Guo, W. “Stock price forecasting with deep learning: A comparative study.” Mathematics 8.9 1441 (2020).
  34. Boukhers, Z., Bouabdallah, A., Lohr, M., Jürjens, J. “Ensemble and multimodal approach for forecasting cryptocurrency price.” arXiv preprint arXiv:2202.08967 .
  35. Stahlschmidt, S. R., Ulfenborg, B., Synnergren, J. “Multimodal deep learning for biomedical data fusion: a review.” Briefings in Bioinformatics 23.2 bbab569 (2022).
  36. Liang, P. P., Zadeh, A., Morency, L.-P. “Foundations and recent trends in multimodal machine learning: Principles, challenges, and open questions.” arXiv preprint arXiv:2209.03430 .
  37. Fu, Z., et al. “A cross-modal fusion network based on self-attention and residual structure for multimodal emotion recognition.” arXiv preprint arXiv:2111.02172 .
  38. Zhou, H., Du, J., Zhang, Y., Wang, Q., Liu, Q.-F., Lee, C.-H. “Information fusion in attention networks using adaptive and multi-level factorized bilinear pooling for audio-visual emotion recognition.” IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 2617–2629 (2021).
  39. Shih, S.-Y., Sun, F.-K., Lee, H.-y. “Temporal pattern attention for multivariate time series forecasting.” Machine Learning 108 1421–1441 (2019).
  40. Lim, B., Arık, S. Ö., Loeff, N., Pfister, T. “Temporal fusion transformers for interpretable multi-horizon time series forecasting.” International Journal of Forecasting 37.4 1748–1764 (2021).
  41. Zhang, H., Zou, Y., Yang, X., Yang, H. “A temporal fusion transformer for short-term freeway traffic speed multistep prediction.” Neurocomputing 500 329–340 (2022).
  42. Hashish, I. A., Forni, F., Andreotti, G., Facchinetti, T., Darjani, S. “A hybrid model for bitcoin prices prediction using hidden Markov models and optimized LSTM networks.” In 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) IEEE 721–728 (2019) .
  43. Costa, D., La Cava, L., Tagarelli, A. “Show me your NFT and I tell you how it will perform: Multimodal representation learning for NFT selling price prediction.” In Proceedings of the ACM Web Conference 2023 1875–1885 (2023) .
  44. Zhang, X., Zhang, L., et al. “Forecasting Method of Stock Market Volatility Based on Multidimensional Data Fusion.” Wireless Communications and Mobile Computing 2022.
  45. Ding, X., Zhang, Y., Liu, T., Duan, J. “Deep learning for event-driven stock prediction.” In Twenty-fourth international joint conference on artificial intelligence 2327–2333 (2015) .
  46. Kulakowski, M. “CryptoBERT.” https://huggingface.co/\\ElKulako/cryptobert.
  47. Hyndman, R. J., Koehler, A. B. “Another look at measures of forecast accuracy.” International journal of forecasting 22.4 679–688 (2006).
  48. Triebe, O., Hewamalage, H., Pilyugina, P., Laptev, N., Bergmeir, C., Rajagopal, R. “NeuralProphet: Explainable Forecasting at Scale.” (2021) 2111.15397.
  49. Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y. “On the properties of neural machine translation: Encoder-decoder approaches.” arXiv preprint arXiv:1409.1259 .
  50. Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., Cottrell, G. “A dual-stage attention-based recurrent neural network for time series prediction.” arXiv preprint arXiv:1704.02971 .
  51. Ranasinghe, H., Halgamuge, M. N. “Twitter sentiment data analysis of user behavior on cryptocurrencies: Bitcoin and ethereum.” In Analyzing Global Social Media Consumption IGI Global 277–291 (2021).
  52. Youssfi Nouira, A., Bouchakwa, M., Jamoussi, Y. “Bitcoin Price Prediction Considering Sentiment Analysis on Twitter and Google News.” In Proceedings of the 27th International Database Engineered Applications Symposium 71–78 (2023) .
  53. Wen, Q., et al. “Transformers in time series: A survey.” arXiv preprint arXiv:2202.07125 .
  54. Kshetri, N. “Policy, ethical, social, and environmental considerations of Web3 and the metaverse.” IT Professional 24.3 4–8 (2022).
  55. Ding, W., et al. “DeSci based on Web3 and DAO: A comprehensive overview and reference model.” IEEE Transactions on Computational Social Systems 9.5 1563–1573 (2022).
  56. Kuznetsov, V., Mohri, M. “Discrepancy-based theory and algorithms for forecasting non-stationary time series.” Annals of Mathematics and Artificial Intelligence 88.4 367–399 (2020).
  57. Arik, S. O., Yoder, N. C., Pfister, T. “Self-adaptive forecasting for improved deep learning on non-stationary time-series.” arXiv preprint arXiv:2202.02403 .
  58. Kaiser, L. “Seasonality in cryptocurrencies.” Finance Research Letters 31.
  59. Wen, Q., Zhang, Z., Li, Y., Sun, L. “Fast RobustSTL: Efficient and robust seasonal-trend decomposition for time series with complex patterns.” In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 2203–2213 (2020) .
  60. Bandara, K., Bergmeir, C., Hewamalage, H. “LSTM-MSNet: Leveraging forecasts on sets of related time series with multiple seasonal patterns.” IEEE transactions on neural networks and learning systems 32.4 1586–1599 (2020).
  61. Dao, T., Fu, D., Ermon, S., Rudra, A., Ré, C. “Flashattention: Fast and memory-efficient exact attention with io-awareness.” Advances in Neural Information Processing Systems 35 16344–16359 (2022).
  62. Koki, C., Leonardos, S., Piliouras, G. “Exploring the predictability of cryptocurrencies via Bayesian hidden Markov models.” Research in International Business and Finance 59 101554 (2022).
  63. Ariyo, A. A., Adewumi, A. O., Ayo, C. K. “Stock price prediction using the ARIMA model.” In 2014 UKSim-AMSS 16th international conference on computer modelling and simulation IEEE 106–112 (2014) .
  64. Ibrahim, A., Kashef, R., Li, M., Valencia, E., Huang, E. “Bitcoin network mechanics: Forecasting the btc closing price using vector auto-regression models based on endogenous and exogenous feature variables.” Journal of Risk and Financial Management 13.9 189 (2020).
  65. Rathan, K., Sai, S. V., Manikanta, T. S. “Crypto-currency price prediction using decision tree and regression techniques.” In 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI) IEEE 190–194 (2019) .
  66. Li, Y., Dai, W. “Bitcoin price forecasting method based on CNN-LSTM hybrid neural network model.” The journal of engineering 2020.13 344–347 (2020).
  67. Serafini, G., et al. “Sentiment-driven price prediction of the bitcoin based on statistical and deep learning approaches.” In 2020 International Joint Conference on Neural Networks (IJCNN) IEEE 1–8 (2020) .
  68. Phaladisailoed, T., Numnonda, T. “Machine learning models comparison for bitcoin price prediction.” In 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE) IEEE 506–511 (2018) .
  69. Kim, T., Kim, H. Y. “Forecasting stock prices with a feature fusion LSTM-CNN model using different representations of the same data.” PloS one 14.2 e0212320 (2019).
  70. Politis, A., Doka, K., Koziris, N. “Ether price prediction using advanced deep learning models.” In 2021 IEEE International Conference on Blockchain and Cryptocurrency (ICBC) IEEE 1–3 (2021) .
  71. Fisher, R. A. “Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population.” Biometrika 10.4 507–521 (1915).
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now