PruneSymNet: A Symbolic Neural Network and Pruning Algorithm for Symbolic Regression (2401.15103v1)
Abstract: Symbolic regression aims to derive interpretable symbolic expressions from data in order to better understand and interpret data. %which plays an important role in knowledge discovery and interpretable machine learning. In this study, a symbolic network called PruneSymNet is proposed for symbolic regression. This is a novel neural network whose activation function consists of common elementary functions and operators. The whole network is differentiable and can be trained by gradient descent method. Each subnetwork in the network corresponds to an expression, and our goal is to extract such subnetworks to get the desired symbolic expression. Therefore, a greedy pruning algorithm is proposed to prune the network into a subnetwork while ensuring the accuracy of data fitting. The proposed greedy pruning algorithm preserves the edge with the least loss in each pruning, but greedy algorithm often can not get the optimal solution. In order to alleviate this problem, we combine beam search during pruning to obtain multiple candidate expressions each time, and finally select the expression with the smallest loss as the final result. It was tested on the public data set and compared with the current popular algorithms. The results showed that the proposed algorithm had better accuracy.
- Symbolic regression via genetic programming. In Proceedings. Vol.1. Sixth Brazilian Symposium on Neural Networks, pages 173–178, 2000. doi: 10.1109/SBRN.2000.889734.
- Neural symbolic regression that scales. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 936–945. PMLR, 18–24 Jul 2021.
- Contemporary symbolic regression methods and their relative performance. In Proceedings of 35th Conference on Neural Information Processing Systems, 2021.
- Dynamic network surgery for efficient dnns. Advances in neural information processing systems, 29, 2016.
- End-to-end symbolic regression with transformers. arXiv:2204.10532, 2022.
- Integration of neural network-based symbolic regression in deep learning for scientific discovery. IEEE Transactions on Neural Networks and Learning Systems, 32(9):4166–4177, 2021. doi: 10.1109/TNNLS.2020.3017010.
- John R. Koza. Genetic programming: on the programming of computers by means of natural selection. MIT, 1992.
- Genetic programming needs better benchmarks. In Terence Soule and Jason H. Moore, editors, Genetic and Evolutionary Computation Conference, GECCO ’12, Philadelphia, PA, USA, July 7-11, 2012, pages 791–798. ACM, 2012. URL https://doi.org/10.1145/2330163.2330273.
- Symbolic regression via neural-guided genetic programming population seeding. arXiv preprint arXiv:2111.00053, 2021.
- Numerical Optimization. Springer, 2006.
- Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. In Proceedings of the 2021 International Conference on Learning Representations, Vienna, Austria, May 2021.
- Xipeng Qiu. Neural Networks and Deep Learning. China machine press, Bei Jing, 2020. ISBN 9787111649687. URL https://nndl.github.io/.
- Learning equations for extrapolation and control. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4442–4450. PMLR, 10–15 Jul 2018.
- Ai feynman: A physics-inspired method for symbolic regression. SCIENCE ADVANCES, 6(16), 2020.
- AI feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity. In Proceedings of the conference and Workshop on Neural Information Processing Systems, December 2020.
- Symbolicgpt: A generative transformer model for symbolic regression. arXiv:2106.14131, 2021.