Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Framework for Empowering Reinforcement Learning Agents with Causal Analysis: Enhancing Automated Cryptocurrency Trading (2310.09462v2)

Published 14 Oct 2023 in cs.AI and cs.LG

Abstract: Despite advances in artificial intelligence-enhanced trading methods, developing a profitable automated trading system remains challenging in the rapidly evolving cryptocurrency market. This research focuses on developing a reinforcement learning (RL) framework to tackle the complexities of trading five prominent altcoins: Binance Coin, Ethereum, Litecoin, Ripple, and Tether. To this end, we present the CausalReinforceNet~(CRN) framework, which integrates both Bayesian and dynamic Bayesian network techniques to empower the RL agent in trade decision-making. We develop two agents using the framework based on distinct RL algorithms to analyse performance compared to the Buy-and-Hold benchmark strategy and a baseline RL model. The results indicate that our framework surpasses both models in profitability, highlighting CRN's consistent superiority, although the level of effectiveness varies across different cryptocurrencies.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. A synchronous deep reinforcement learning model for automated multi-stock trading. Progress in Artificial Intelligence, 10(1):83–97, 2021.
  2. Review on reinforcement learning, research evolution and scope of application. In 2021 5th international conference on computing methodologies and communication (ICCMC), pages 1416–1423. IEEE, 2021.
  3. An analysis of paper money and the concept of multi-genus banknotes. Financial and Accounting Principles in Islamic Finance, pages 49–73, 2019.
  4. Convolution on neural networks for high-frequency trend prediction of cryptocurrency exchange rates using technical indicators. Expert Systems with Applications, 149:113250, 2020.
  5. Intelligent algorithmic trading strategy using reinforcement learning and directional change. IEEE Access, 9:114659–114671, 2021.
  6. Applying artificial intelligence in cryptocurrency markets: A survey. Algorithms, 15(11):428, 2022.
  7. A deep reinforcement learning-based decision support system for automated stock market trading. IEEE Access, 10:127469–127501, 2022.
  8. Reinforcement learning based on state space model using growing neural gas for a mobile robot. In 2018 Joint 10th International Conference on Soft Computing and Intelligent Systems (SCIS) and 19th International Symposium on Advanced Intelligent Systems (ISIS), pages 1410–1413. IEEE, 2018.
  9. Attention-aware face hallucination via deep reinforcement learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 690–698, 2017.
  10. A q-learning agent for automated trading in equity stock markets. Expert Systems with Applications, 163:113761, 2021.
  11. Reinforcement learning-based simulation and automation for tower crane 3d lift planning. Automation in Construction, 144:104620, 2022.
  12. Leonardo Conegundes and Adriano C Machado Pereira. Beating the stock market with a deep reinforcement learning day trading system. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2020.
  13. Towards stochastic modeling for two-phase flow interfacial area predictions: A physics-informed reinforcement learning approach. International Journal of Heat and Mass Transfer, 192:122919, 2022.
  14. Reinforcement learning in surgery. Surgery, 170(1):329–332, 2021.
  15. Artificial intelligence and marketing: Pitfalls and opportunities. Journal of Interactive Marketing, 51(1):91–105, 2020.
  16. On deep reinforcement learning for static routing and wavelength assignment. IEEE Journal of Selected Topics in Quantum Electronics, 28(4):1–12, 2022.
  17. Automated liquidity provision. Pacific-Basin Finance Journal, 45:1–13, 2017.
  18. Deep reinforcement learning for cryptocurrency trading: Practical approach to address backtest overfitting. arXiv preprint arXiv:2209.05559, 2022.
  19. How to enable automated trading engines to cope with news-related liquidity shocks? extracting signals from unstructured data. Decision Support Systems, 62:32–42, 2014.
  20. Douglas AE Harewood-Gill. Q-Routing for Enhanced Performance Within a SDN Controlled Network. PhD thesis, University of Bristol, 2022.
  21. Ethereum vs. bitcoin. URL: www. economist. com, 2016.
  22. Exploiting generalization in the subspaces for faster model-based reinforcement learning. IEEE transactions on neural networks and learning systems, 30(6):1635–1650, 2018.
  23. Toward a framework for safeguarding financial stability, volume 4. International Monetary Fund Washington, DC, 2004.
  24. Automated trading systems statistical and machine learning methods and hardware implementation: a survey. Enterprise Information Systems, 13(1):132–144, 2019.
  25. Body calibration: Automatic inter-task mapping between multi-legged robots with different embodiments in transfer reinforcement learning. In Actuators, volume 11, page 140. MDPI, 2022.
  26. Reinforcement learning with unsupervised auxiliary tasks. arXiv preprint arXiv:1611.05397, 2016.
  27. Adaptive supply chain: Demand–supply synchronization using deep reinforcement learning. Algorithms, 14(8):240, 2021.
  28. An empirical study of representation learning for reinforcement learning in healthcare. arXiv preprint arXiv:2011.11235, 2020.
  29. Combining deep reinforcement learning with technical analysis and trend monitoring on cryptocurrency markets. Neural Computing and Applications, pages 1–18, 2023.
  30. Cryptocurrency trading using machine learning. Journal of Risk and Financial Management, 13(8):178, 2020.
  31. Empirical analysis of automated stock trading using deep reinforcement learning. Applied Sciences, 13(1):633, 2023.
  32. Exploration in deep reinforcement learning: A survey. Information Fusion, 85:1–22, 2022.
  33. Yujun Lai. Towards Personalised Robotic Assessment and Response during Physical Human Robot Interactions. PhD thesis, 2022.
  34. Woonghee Lee. Reward-based participant selection for improving federated reinforcement learning. ICT Express, 2022.
  35. Stock market forecasting using deep learning and technical analysis: a systematic review. IEEE access, 8:185232–185242, 2020.
  36. Efficient experience replay based deep deterministic policy gradient for agc dispatch in integrated energy system. Applied energy, 285:116386, 2021.
  37. Enhancing a stock timing strategy by reinforcement learning. IAENG Int. J. Comput. Sci, 48:1–10, 2021.
  38. Neural symbolic machines: Learning semantic parsers on freebase with weak supervision. arXiv preprint arXiv:1611.00020, 2016.
  39. Michael L Littman. Reinforcement learning improves behaviour from evaluative feedback. Nature, 521(7553):445–451, 2015.
  40. Ge Liu and Wenping Ma. A quantum artificial neural network for stock closing price prediction. Information Sciences, 598:75–85, 2022.
  41. Balancing accuracy and fairness for interactive recommendation with reinforcement learning. arXiv preprint arXiv:2106.13386, 2021.
  42. A deep reinforcement learning approach for automated cryptocurrency trading. In Artificial Intelligence Applications and Innovations: 15th IFIP WG 12.5 International Conference, AIAI 2019, Hersonissos, Crete, Greece, May 24–26, 2019, Proceedings 15, pages 247–258. Springer, 2019.
  43. A deep q-learning portfolio management framework for the cryptocurrency market. Neural Computing and Applications, 32(23):17229–17244, 2020.
  44. An empirical study of machine learning algorithms for stock daily trading strategy. Mathematical problems in engineering, 2019, 2019.
  45. A comparison of resampling methods for remote sensing classification and accuracy assessment. Remote Sensing of Environment, 208:145–153, 2018.
  46. A parallel multi-module deep reinforcement learning algorithm for stock trading. Neurocomputing, 449:290–302, 2021.
  47. Arvind Matharu. Understanding cryptocurrencies: The money of the future. Business Expert Press, 2018.
  48. Learning humanoid robot running motions with symmetry incentive through proximal policy optimization. Journal of Intelligent & Robotic Systems, 102(3):54, 2021.
  49. Disruptive innovation of cryptocurrencies in consumer acceptance and trust. Information Systems and e-Business Management, 17:195–222, 2019.
  50. Learning to navigate in complex environments. arXiv preprint arXiv:1611.03673, 2016.
  51. Satoshi Nakamoto and A Bitcoin. A peer-to-peer electronic cash system. Bitcoin.–URL: https://bitcoin. org/bitcoin. pdf, 4(2), 2008.
  52. Cristian Păuna. A prediction model using the price cyclicality function optimized for algorithmic trading in financial market. International Journal of Computer and Information Engineering, 13(4):222–230, 2019.
  53. An overview of machine learning, deep learning, and reinforcement learning-based techniques in quantitative finance: Recent progress and challenges. Applied Sciences, 13(3):1956, 2023.
  54. Recommending cryptocurrency trading points with deep reinforcement learning approach. Applied Sciences, 10(4):1506, 2020.
  55. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
  56. Deterministic policy gradient algorithms. In International conference on machine learning, pages 387–395. Pmlr, 2014.
  57. Deep neural network and time series approach for finance systems: predicting the movement of the indian stock market. Journal of Organizational and End User Computing (JOEUC), 33(5):204–226, 2021.
  58. An optimal deep learning-based lstm for stock price prediction using twitter sentiment analysis. Applied Intelligence, 52(12):13675–13688, 2022.
  59. A hardware implementation for deep reinforcement learning machine. In 2022 RIVF International Conference on Computing and Communication Technologies (RIVF), pages 209–213. IEEE, 2022.
  60. Controlling distributed energy resources via deep reinforcement learning for load flexibility and energy efficiency. Applied Energy, 304:117733, 2021.
  61. Optimizing automated trading systems with deep reinforcement learning. Algorithms, 16(1):23, 2023.
  62. Deep reinforcement learning-based rescue resource distribution scheduling of storm surge inundation emergency logistics. IEEE Transactions on Industrial Informatics, 2023.
  63. Portfolio trading system of digital currencies: A deep reinforcement learning with multidimensional attention gating mechanism. Neurocomputing, 402:171–182, 2020.
  64. Portfolio management system in equity market neutral using reinforcement learning. Applied Intelligence, pages 1–13, 2021.
  65. A dynamic power allocation scheme in power-domain noma using actor-critic reinforcement learning. In 2018 IEEE/CIC International Conference on Communications in China (ICCC), pages 719–723. IEEE, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Rasoul Amirzadeh (3 papers)
  2. Dhananjay Thiruvady (14 papers)
  3. Asef Nazari (12 papers)
  4. Mong Shan Ee (3 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now