Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization (2403.16667v1)
Abstract: Portfolio optimization involves determining the optimal allocation of portfolio assets in order to maximize a given investment objective. Traditionally, some form of mean-variance optimization is used with the aim of maximizing returns while minimizing risk, however, more recently, deep reinforcement learning formulations have been explored. Increasingly, investors have demonstrated an interest in incorporating ESG objectives when making investment decisions, and modifications to the classical mean-variance optimization framework have been developed. In this work, we study the use of deep reinforcement learning for responsible portfolio optimization, by incorporating ESG states and objectives, and provide comparisons against modified mean-variance approaches. Our results show that deep reinforcement learning policies can provide competitive performance against mean-variance approaches for responsible portfolio allocation across additive and multiplicative utility functions of financial and ESG responsibility objectives.
- A guide to ESG portfolio construction. The Journal of Portfolio Management, 45(4): 61–66.
- Reinforcement learning in economics and finance. Computational Economics, 1–38.
- Social responsibility portfolio optimization incorporating ESG criteria. Journal of Management Science and Engineering, 6(1): 75–85.
- Optimization methods in finance, volume 5. Cambridge University Press.
- Dinkelbach, W. 1967. On nonlinear fractional programming. Management science, 13(7): 492–498.
- Estrada, J. 2008. Mean-semivariance optimization: A heuristic approach. Journal of Applied Finance (Formerly Financial Practice and Education), 18(1).
- The wages of social responsibility—where are they? A critical review of ESG investing. Review of Financial Economics, 26: 25–35.
- Recent advances in reinforcement learning in finance. Mathematical Finance, 33(3): 437–503.
- Portfolio selection with higher moments. Quantitative Finance, 10(5): 469–485.
- Integrating ESG in portfolio construction. The Journal of Portfolio Management, 45(4): 67–81.
- A note on the sensitivity of the strategic asset allocation problem. Operations Research Perspectives, 2: 133–136.
- A deep reinforcement learning framework for the financial portfolio management problem. arXiv preprint arXiv:1706.10059.
- Jin, I. 2022. ESG-screening and factor-risk-adjusted performance: The concentration level of screening does matter. Journal of Sustainable Finance & Investment, 12(4): 1125–1145.
- FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning. Advances in Neural Information Processing Systems, 35: 1835–1849.
- FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance. arXiv preprint arXiv:2011.09607.
- Portfolio optimization with linear and fixed transaction costs. Annals of Operations Research, 152: 341–365.
- Markowitz, H. 1952. Portfolio Selection. The Journal of Finance, 7(1): 77–91.
- Reinforcement learning for trading. Advances in Neural Information Processing Systems, 11.
- Performance functions and reinforcement learning for trading systems and portfolios. Journal of forecasting, 17(5-6): 441–470.
- Ouchen, A. 2022. Is the ESG portfolio less turbulent than a market benchmark portfolio? Risk Management, 24(1): 1–33.
- Deep learning for financial applications: A survey. Applied Soft Computing, 93: 106384.
- Responsible investing: The ESG-efficient frontier. Journal of Financial Economics, 142(2): 572–597.
- On imposing ESG constraints of portfolio selection for sustainable investment and comparing the efficient frontiers in the weight space. Sage Open, 10(4): 2158244020975070.
- Stable-baselines3: Reliable reinforcement learning implementations. The Journal of Machine Learning Research, 22(1): 12348–12355.
- Schaible, S. 1974. Parameter-free convex equivalent and dual programs of fractional programming problems. Zeitschrift für Operations Research, 18: 187–196.
- High-Dimensional Continuous Control Using Generalized Advantage Estimation. In Proceedings of the International Conference on Learning Representations (ICLR).
- Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
- Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization.
- Reinforcement Learning: An Introduction. The MIT Press, second edition.
- Policy Gradient Methods for Reinforcement Learning with Function Approximation. In Solla, S.; Leen, T.; and Müller, K., eds., Advances in Neural Information Processing Systems, volume 12. MIT Press.
- ESG for all? The impact of ESG screening on return, risk, and diversification. Journal of Applied Corporate Finance, 28(2): 47–55.
- Reinforcement-learning based portfolio management with augmented asset movement prediction states. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 1112–1119.
- Model-based deep reinforcement learning for dynamic portfolio optimization. arXiv preprint arXiv:1901.08740.