GFN-SR: Symbolic Regression with Generative Flow Networks (2312.00396v1)
Abstract: Symbolic regression (SR) is an area of interpretable machine learning that aims to identify mathematical expressions, often composed of simple functions, that best fit in a given set of covariates $X$ and response $y$. In recent years, deep symbolic regression (DSR) has emerged as a popular method in the field by leveraging deep reinforcement learning to solve the complicated combinatorial search problem. In this work, we propose an alternative framework (GFN-SR) to approach SR with deep learning. We model the construction of an expression tree as traversing through a directed acyclic graph (DAG) so that GFlowNet can learn a stochastic policy to generate such trees sequentially. Enhanced with an adaptive reward baseline, our method is capable of generating a diverse set of best-fitting expressions. Notably, we observe that GFN-SR outperforms other SR algorithms in noisy data regimes, owing to its ability to learn a distribution of rewards over a space of candidate solutions.
- “Flow network based generative models for non-iterative diverse candidate generation” In Advances in Neural Information Processing Systems 34, 2021, pp. 27381–27394
- “Gflownet foundations” In arXiv preprint arXiv:2111.09266, 2021
- “Overview of the Transformer-based Models for NLP Tasks” In 2020 15th Conference on Computer Science and Information Systems (FedCSIS), 2020, pp. 179–183 IEEE
- “An improved genetic algorithm with a new initialization mechanism based on regression techniques” In Information 9.7 MDPI, 2018, pp. 167
- “Long short-term memory” In Neural computation 9.8 MIT press, 1997, pp. 1735–1780
- “Bayesian Symbolic Regression”, 2020 arXiv:1910.08892 [stat.ME]
- Diederik P Kingma and Jimmy Ba “Adam: A method for stochastic optimization” In arXiv preprint arXiv:1412.6980, 2014
- John R Koza “Hierarchical genetic algorithms operating on populations of computer programs.” In IJCAI 89, 1989, pp. 768–774
- “Contemporary symbolic regression methods and their relative performance” In arXiv preprint arXiv:2107.14351, 2021
- “A unified framework for deep symbolic regression” In Advances in Neural Information Processing Systems 35, 2022, pp. 33985–33998
- “Trajectory balance: Improved credit assignment in gflownets” In Advances in Neural Information Processing Systems 35, 2022, pp. 5955–5967
- Sebastian Musslick “Recovering Quantitative Models of Human Information Processing with Differentiable Architecture Search.” In Proceedings of the Annual Meeting of the Cognitive Science Society 43 eScholarship, 2021 URL: https://escholarship.org/uc/item/9wd571ts
- “Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients” In arXiv preprint arXiv:1912.04871, 2019
- “Policy gradient methods for reinforcement learning with function approximation” In Advances in neural information processing systems 12, 1999
- Wassim Tenachi, Rodrigo Ibata and Foivos I Diakogiannis “Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws” In arXiv preprint arXiv:2303.03192, 2023
- “AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity” In Advances in Neural Information Processing Systems 33, 2020, pp. 4860–4871
- “Semantically-based crossover in genetic programming: application to real-valued symbolic regression” In Genetic Programming and Evolvable Machines 12 Springer, 2011, pp. 91–119
- Greg Van Houdt, Carlos Mosquera and Gonzalo Nápoles “A review on the long short-term memory model” In Artificial Intelligence Review 53 Springer, 2020, pp. 5929–5955
- “Attention is all you need” In Advances in neural information processing systems 30, 2017
- Marco Virgolin and Solon P Pissis “Symbolic regression is np-hard” In arXiv preprint arXiv:2207.01018, 2022
- Ekaterina Yurievna Vladislavleva “Model-based problem solving through symbolic regression via pareto genetic programming” CentER, Tilburg University Tilburg, 2008
- “The optimal reward baseline for gradient-based reinforcement learning” In arXiv preprint arXiv:1301.2315, 2013