Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
143 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Generative Adversarial Symmetry Discovery (2302.00236v4)

Published 1 Feb 2023 in cs.LG

Abstract: Despite the success of equivariant neural networks in scientific applications, they require knowing the symmetry group a priori. However, it may be difficult to know which symmetry to use as an inductive bias in practice. Enforcing the wrong symmetry could even hurt the performance. In this paper, we propose a framework, LieGAN, to automatically discover equivariances from a dataset using a paradigm akin to generative adversarial training. Specifically, a generator learns a group of transformations applied to the data, which preserve the original distribution and fool the discriminator. LieGAN represents symmetry as interpretable Lie algebra basis and can discover various symmetries such as the rotation group $\mathrm{SO}(n)$, restricted Lorentz group $\mathrm{SO}(1,3)+$ in trajectory prediction and top-quark tagging tasks. The learned symmetry can also be readily used in several existing equivariant neural networks to improve accuracy and generalization in prediction.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Noether networks: meta-learning useful conserved quantities. Advances in Neural Information Processing Systems, 34:16384–16397, 2021.
  2. Data augmentation generative adversarial networks. arXiv preprint arXiv:1711.04340, 2017.
  3. Bekkers, E. J. B-spline cnns on lie groups. arXiv preprint arXiv:1909.12057, 2019.
  4. Learning invariances in neural networks from training data. Advances in neural information processing systems, 33:17605–17616, 2020.
  5. 970 million druglike small molecules for virtual screening in the chemical universe database gdb-13. Journal of the American Chemical Society, 131(25):8732–8733, 2009.
  6. Group equivariant convolutional networks. In International conference on machine learning, pp. 2990–2999. PMLR, 2016.
  7. Gauge equivariant convolutional networks and the icosahedral cnn. In International conference on Machine learning, pp. 1321–1330. PMLR, 2019a.
  8. Spherical cnns. arXiv preprint arXiv:1801.10130, 2018.
  9. A general theory of equivariant cnns on homogeneous spaces. Advances in neural information processing systems, 32, 2019b.
  10. A kernel theory of modern data augmentation. In International Conference on Machine Learning, pp. 1528–1537. PMLR, 2019.
  11. Automatic symmetry discovery with lie algebra convolutional network. Advances in Neural Information Processing Systems, 34:2503–2515, 2021.
  12. Deng, L. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 29(6):141–142, 2012.
  13. Symmetrygan: Symmetry discovery with deep learning. arXiv preprint arXiv:2112.05722, 2021.
  14. Reparameterizing distributions on lie groups. In The 22nd International Conference on Artificial Intelligence and Statistics, pp.  3244–3253. PMLR, 2019.
  15. Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data. In International Conference on Machine Learning, pp. 3165–3176. PMLR, 2020.
  16. A practical method for constructing equivariant multilayer perceptrons for arbitrary matrix groups. In International Conference on Machine Learning, pp. 3318–3328. PMLR, 2021.
  17. An efficient lorentz equivariant graph neural network for jet tagging. Journal of High Energy Physics, 2022(7), jul 2022. doi: 10.1007/jhep07(2022)030. URL https://doi.org/10.1007%2Fjhep07%282022%29030.
  18. Generative adversarial networks. Proceedings of the International Conference on Neural Information Processing Systems (NIPS), pp.  2672–2680, 2014.
  19. Hamiltonian neural networks. Advances in neural information processing systems, 32, 2019.
  20. Image-to-image translation with conditional adversarial networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.  5967–5976, 2016.
  21. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  4401–4410, 2019.
  22. Top quark tagging reference dataset, March 2019. URL https://doi.org/10.5281/zenodo.2603256.
  23. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  24. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  25. Lie groups beyond an introduction, volume 140. Springer, 1996.
  26. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International Conference on Machine Learning, pp. 2747–2755. PMLR, 2018.
  27. Detecting symmetries with neural networks. Machine Learning: Science and Technology, 2(1):015010, 2020.
  28. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90, 2017.
  29. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
  30. Liegg: Studying learned lie group generators. arXiv preprint arXiv:2210.04345, 2022.
  31. f-gan: Training generative neural samplers using variational divergence minimization. Advances in neural information processing systems, 29, 2016.
  32. Learning partial equivariances from data. arXiv preprint arXiv:2110.10211, 2021.
  33. Fast and accurate modeling of molecular atomization energies with machine learning. Physical review letters, 108(5):058301, 2012.
  34. E (n) equivariant graph neural networks. In International conference on machine learning, pp. 9323–9332. PMLR, 2021.
  35. Incorporating symmetry into deep dynamics models for improved generalization. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=wta_8Hx2KD.
  36. Approximately equivariant networks for imperfectly symmetric dynamics. arXiv preprint arXiv:2201.11969, 2022.
  37. General e (2)-equivariant steerable cnns. Advances in Neural Information Processing Systems, 32, 2019.
  38. Deep scale-spaces: Equivariance over scale. Advances in Neural Information Processing Systems, 32, 2019.
  39. Deep sets. Advances in neural information processing systems, 30, 2017.
  40. Meta-learning symmetries by reparameterization. arXiv preprint arXiv:2007.02933, 2020.
  41. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp.  2223–2232, 2017.
Citations (17)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now