Symbolic Equation Solving via Reinforcement Learning (2401.13447v2)
Abstract: Machine-learning methods are gradually being adopted in a wide variety of social, economic, and scientific contexts, yet they are notorious for struggling with exact mathematics. A typical example is computer algebra, which includes tasks like simplifying mathematical terms, calculating formal derivatives, or finding exact solutions of algebraic equations. Traditional software packages for these purposes are commonly based on a huge database of rules for how a specific operation (e.g., differentiation) transforms a certain term (e.g., sine function) into another one (e.g., cosine function). These rules have usually needed to be discovered and subsequently programmed by humans. Efforts to automate this process by machine-learning approaches are faced with challenges like the singular nature of solutions to mathematical problems, when approximations are unacceptable, as well as hallucination effects leading to flawed reasoning. We propose a novel deep-learning interface involving a reinforcement-learning agent that operates a symbolic stack calculator to explore mathematical relations. By construction, this system is capable of exact transformations and immune to hallucination. Using the paradigmatic example of solving linear equations in symbolic form, we demonstrate how our reinforcement-learning agent autonomously discovers elementary transformation rules and step-by-step solutions.
- Deep reinforcement learning: A brief survey. IEEE Signal Process Mag., 34(6):26--38, 2017. doi: 10.1109/MSP.2017.2743240.
- HOList: An environment for machine learning of higher order logic theorem proving. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 454--463, 2019. URL https://proceedings.mlr.press/v97/bansal19a.html.
- Sparks of artificial general intelligence: Early experiments with GPT-4, 2023. URL https://arxiv.org/abs/2303.12712.
- Charles Q. Choi. 7 revealing ways AIs fail: Neural networks can be disastrously brittle, forgetful, and surprisingly bad at math. IEEE Spectr., 58(10):42--47, 2021. doi: 10.1109/MSPEC.2021.9563958.
- Deep symbolic regression for recurrent sequences, 2022. URL https://arxiv.org/abs/2201.04600.
- Advancing mathematics by guiding human intuition with AI. Nature, 600(7887):70--74, 2021. doi: 10.1038/s41586-021-04086-x.
- Ernest Davis. The use of deep learning for symbolic integration: A review of (Lample and Charton, 2019), 2019. URL https://arxiv.org/abs/1912.05752.
- A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. Proc. Natl. Acad. Sci., 119(32):e2123433119, 2022. doi: 10.1073/pnas.2123433119.
- Discovering faster matrix multiplication algorithms with reinforcement learning. Nature, 610(7930):47--53, 2022. doi: 10.1038/s41586-022-05172-4.
- Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci., 115(34):8505--8510, 2018. doi: 10.1073/pnas.1718942115.
- Deep reinforcement learning with double q-learning. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, page 2094–2100, 2016.
- Yang-Hui He. Machine-learning mathematical structures. International Journal of Data Science in the Mathematical Sciences, 01(01):23--47, 2023. doi: 10.1142/S2810939222500010.
- Reinforcement learning of theorem proving. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, page 8836–8847, 2018.
- Deep learning for symbolic mathematics. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=S1eZYeHFDS.
- Solving quantitative reasoning problems with language models, 2022. URL https://arxiv.org/abs/2206.14858.
- Solving nonlinear equations systems with an enhanced reinforcement learning based differential evolution. Complex System Modeling and Simulation, 2(1):78--95, 2022. doi: 10.23919/CSMS.2022.0003.
- F. Marquardt. Machine learning and quantum devices. Lecture Notes for the 2019 Les Houches Summer School, 2020.
- Sympy: symbolic computing in Python. PeerJ Comput. Sci., 3:e103, 2017. doi: 10.7717/peerj-cs.103.
- Human-level control through deep reinforcement learning. Nature, 518(7540):529--533, 2015. doi: 10.1038/nature14236.
- Stephen Ornes. Symbolic mathematics finally yields to neural networks. Quanta Magazine, 2020. URL https://www.quantamagazine.org/symbolic-mathematics-finally-yields-to-neural-networks-20200520/.
- Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32, 2019. URL https://proceedings.neurips.cc/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf.
- Contrastive reinforcement learning of symbolic reasoning domains. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL https://openreview.net/forum?id=ZarM_uLVyGw.
- Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, second edition, 2018.
- AI Feynman: A physics-inspired method for symbolic regression. Sci. Adv., 6(16), 2020. doi: 10.1126/sciadv.aay2631.
- Deep reinforcement learning and the deadly triad, 2018.
- Convergent and efficient deep Q learning algorithm. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=OJm3HZuj4r7.
- Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=_VjQlMeSB_J.
- Stephen Wolfram. ChatGPT gets its ‘‘Wolfram superpowers’’! Stephen Wolfram Writings, 2023a. URL https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/.
- Stephen Wolfram. Wolfram||||Alpha as the way to bring computational knowledge superpowers to ChatGPT. Stephen Wolfram Writings, 2023b. URL https://writings.stephenwolfram.com/2023/01/wolframalpha-as-the-way-to-bring-computational-knowledge-superpowers-to-chatgpt/.