Optimizing Trading Strategies in Quantitative Markets using Multi-Agent Reinforcement Learning (2303.11959v2)
Abstract: Quantitative markets are characterized by swift dynamics and abundant uncertainties, making the pursuit of profit-driven stock trading actions inherently challenging. Within this context, reinforcement learning (RL), which operates on a reward-centric mechanism for optimal control, has surfaced as a potentially effective solution to the intricate financial decision-making conundrums presented. This paper delves into the fusion of two established financial trading strategies, namely the constant proportion portfolio insurance (CPPI) and the time-invariant portfolio protection (TIPP), with the multi-agent deep deterministic policy gradient (MADDPG) framework. As a result, we introduce two novel multi-agent RL (MARL) methods, CPPI-MADDPG and TIPP-MADDPG, tailored for probing strategic trading within quantitative markets. To validate these innovations, we implemented them on a diverse selection of 100 real-market shares. Our empirical findings reveal that the CPPI-MADDPG and TIPP-MADDPG strategies consistently outpace their traditional counterparts, affirming their efficacy in the realm of quantitative trading.
- “A comprehensive survey on deep neural networks for stock market: The need, challenges, and future directions,” Expert Systems with Applications, vol. 177, pp. 114800, 2021.
- “Quantitative model of price diffusion and market friction based on trading as a mechanistic random process,” Physical Review Letters, vol. 90, no. 10, pp. 108102, 2003.
- Quantitative trading: Algorithms, Analytics, Data, Models, Optimization, Chapman and Hall/CRC, 2017.
- “Time series momentum,” Journal of financial economics, vol. 104, no. 2, pp. 228–250, 2012.
- Huyên Pham, Continuous-time Stochastic Control and Optimization with Financial Applications, vol. 61, Springer Science & Business Media, 2009.
- “An application of stochastic control theory to financial economics,” SIAM Journal on Control and Optimization, vol. 43, no. 2, pp. 502–531, 2004.
- “Deep reinforcement learning for quantitative trading: Challenges and opportunities,” IEEE Intelligent Systems, vol. 37, no. 2, pp. 23–26, 2022.
- Reinforcement learning: An introduction, MIT press, 2018.
- “Diversity-driven knowledge distillation for financial trading using deep reinforcement learning,” Neural Networks, vol. 140, pp. 193–202, 2021.
- “Hypergraph-based reinforcement learning for stock portfolio selection,” in 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 4028–4032.
- “A deep deterministic policy gradient-based strategy for stocks portfolio management,” in 2021 IEEE 6th International Conference on Big Data Analytics (ICBDA), 2021, pp. 230–238.
- “Multi-agent reinforcement learning: A selective overview of theories and algorithms,” Handbook of Reinforcement Learning and Control, pp. 321–384, 2021.
- “A comprehensive survey of multiagent reinforcement learning,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 38, no. 2, pp. 156–172, 2008.
- “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015.
- “Grandmaster level in starcraft ii using multi-agent reinforcement learning,” Nature, vol. 575, no. 7782, pp. 350–354, 2019.
- “MAPS: Multi-agent reinforcement learning-based portfolio management system,” arXiv preprint arXiv:2007.05402, 2020.
- “Modelling stock markets by multi-agent reinforcement learning,” Computational Economics, vol. 57, no. 1, pp. 113–147, 2021.
- “Effectiveness of cppi strategies under discrete-time trading,” Journal of Economic Dynamics and Control, vol. 33, no. 1, pp. 204–220, 2009.
- “Multi-agent actor-critic for mixed cooperative-competitive environments,” in 31th Conference on Neural Information Processing Systems (NeurIPS), 2017, vol. 30.
- “FinRL: Deep reinforcement learning framework to automate trading in quantitative finance,” ACM International Conference on AI in Finance (ICAIF), 2021.