Papers
Topics
Authors
Recent
2000 character limit reached

Dimensionless machine learning: Imposing exact units equivariance (2204.00887v2)

Published 2 Apr 2022 in stat.ML, cs.LG, and physics.data-an

Abstract: Units equivariance (or units covariance) is the exact symmetry that follows from the requirement that relationships among measured quantities of physics relevance must obey self-consistent dimensional scalings. Here, we express this symmetry in terms of a (non-compact) group action, and we employ dimensional analysis and ideas from equivariant machine learning to provide a methodology for exactly units-equivariant machine learning: For any given learning task, we first construct a dimensionless version of its inputs using classic results from dimensional analysis, and then perform inference in the dimensionless space. Our approach can be used to impose units equivariance across a broad range of machine learning methods which are equivariant to rotations and other groups. We discuss the in-sample and out-of-sample prediction accuracy gains one can obtain in contexts like symbolic regression and emulation, where symmetry is important. We illustrate our approach with simple numerical examples involving dynamical systems in physics and ecology.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (83)
  1. Martin Arjovsky. Out of distribution generalization in machine learning. PhD thesis, New York University, 2020.
  2. Henry S Baird. Document image defect models. In Structured Document Image Analysis, pages 546–556. Springer, 1992.
  3. Dimensionally consistent learning with buckingham pi. arXiv:2202.04643, 2022.
  4. Grigory Isaakovich Barenblatt. Scaling and transformation groups. Renormalization group, page 161–180. Cambridge Texts in Applied Mathematics. Cambridge University Press, 1996. doi: 10.1017/CBO9781107050242.009.
  5. Se (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. arXiv:2101.03164, 2021.
  6. A theory of learning from different domains. Machine learning, 79(1):151–175, 2010.
  7. Learning invariances in neural networks. arXiv:2010.11882, 2020.
  8. A volumetric deep convolutional neural network for simulation of mock dark matter halo catalogues. Monthly Notices of the Royal Astronomical Society, 482(3):2861–2871, 2019.
  9. On the sample complexity of learning under geometric stability. Advances in Neural Information Processing Systems, 34, 2021.
  10. Equivariant maps from invariant functions. arXiv preprint arXiv:2209.14991, 2022.
  11. Niels Bohr. I. on the constitution of atoms and molecules. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 26(151):1–25, 1913.
  12. Invariance, encodings, and generalization: learning identity effects with neural networks. arXiv preprint arXiv:2101.08386, 2021.
  13. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences, 113(15):3932–3937, 2016.
  14. Edgar Buckingham. On physically similar systems; illustrations of the use of dimensional equations. Physical Review, 4(4):345–376, 1914.
  15. Lie PCA: Density estimation for symmetric manifolds. arXiv:2008.04278, 2020.
  16. Realistic on-the-fly outcomes of planetary collisions: Machine learning applied to simulations of giant impacts. Astrophysical Journal, 875(1):40, April 2019.
  17. The effect of inefficient accretion on planetary differentiation. Planetary Science Journal, 2(3):93, June 2021.
  18. SE(3)-equivariant self-attention via invariant features. Machine Learning for Physics NeurIPS Workshop, 2022.
  19. A group-theoretic framework for data augmentation. Advances in Neural Information Processing Systems, 33:21321–21333, 2020a.
  20. Supervised community detection with line graph neural networks. Internation Conference on Learning Representations, 2019a.
  21. On the equivalence between graph isomorphism testing and function approximation with gnns. In Advances in Neural Information Processing Systems, pages 15894–15902, 2019b.
  22. Can graph neural networks count substructures? Advances in neural information processing systems, 2020b.
  23. Group equivariant convolutional networks. In Proceedings of the 33rd International Conference on International Conference on Machine Learning, volume 48, page 2990–2999, 2016.
  24. Steerable cnns. In International Conference on Learning Representations (ICLR), 2017.
  25. Spherical cnns, 2018.
  26. Data-driven dimensional analysis: Algorithms for unique and relevant dimensionless groups. arXiv:1708.04303, 2017.
  27. Autoaugment: Learning augmentation policies from data. arXiv:1805.09501, 2018.
  28. Pattern selection and hysteresis in the rietkerk model for banded vegetation in semi-arid environments. Journal of The Royal Society Interface, 11(99):20140465, 2014.
  29. A kernel theory of modern data augmentation. In International Conference on Machine Learning, pages 1528–1537. PMLR, 2019.
  30. Out-of-distribution detection using kernel density polytopes. arXiv:2201.13001, 2022.
  31. Convolutional networks on graphs for learning molecular fingerprints. In Advances in neural information processing systems, pages 2224–2232, 2015.
  32. Bryn Elesedy. Provably strict generalisation benefit for invariance in kernel methods. arXiv:2106.02346, 2021.
  33. Provably strict generalisation benefit for equivariant models. arXiv:2102.10333, 2021.
  34. On the parameter combinations that matter and on those that do not. arXiv:2110.06717, 2021.
  35. A practical method for constructing equivariant multilayer perceptrons for arbitrary matrix groups. arXiv:2104.09459, 2021.
  36. Buckingham theorem application to machine learning algorithms: methodology and practical examples. Master’s thesis, Politecnico di Milano, 2019.
  37. Se (3)-transformers: 3d roto-translation equivariant attention networks. Advances in Neural Information Processing Systems, 33, 2020.
  38. Representation theory: a first course, volume 129. Springer Science & Business Media, 2013.
  39. Graphs, convolutions, and neural networks: From graph filters to graph neural networks. IEEE Signal Processing Magazine, 37(6):128–138, 2020.
  40. Towards a theory of out-of-distribution learning. arXiv:2109.14501, 2021.
  41. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1263–1272. JMLR. org, 2017.
  42. Hamiltonian neural networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32, 2019.
  43. Lorentz-and permutation-invariants of particles. Journal of Physics A: Mathematical and Theoretical, 54(15):155201, 2021.
  44. Ward Haddadin. Invariant polynomials and machine learning. arXiv:2104.12733, 2021.
  45. On the value of target data in transfer learning. Advances in Neural Information Processing Systems, 32, 2019.
  46. Learning to predict the cosmological structure formation. Proceedings of the National Academy of Sciences, 116(28):13825–13832, 2019.
  47. A short tutorial on the weisfeiler-lehman test and its variants. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8533–8537. IEEE, 2021.
  48. Rational invariants of scalings from Hermite normal forms. In Proceedings of the 37th International Symposium on Symbolic and Algebraic Computation, pages 219–226, 2012.
  49. Composing molecules with multiple property constraints. arXiv:2002.03244, 2020.
  50. Transferable meta learning across domains. In UAI, pages 177–187, 2018.
  51. Physics-informed machine learning: Case studies for weather and climate modelling. Philosophical Transactions of the Royal Society A, 379(2194):20200093, 2021.
  52. Super-resolution emulator of cosmological simulations using deep physical models. Monthly Notices of the Royal Astronomical Society, 495(4):4227–4236, 2020.
  53. Risi Kondor. N-body networks: a covariant hierarchical neural network architecture for learning atomic potentials. arXiv:1803.01588, 2018.
  54. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541–551, 1989.
  55. Learning to generalize: Meta-learning for domain generalization. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
  56. Metashift: A dataset of datasets for evaluating contextual distribution shifts and training conflicts. arXiv:2202.06523, 2022.
  57. Domain adaptation: Learning bounds and algorithms. arXiv:0902.3430, 2009.
  58. Invariant and equivariant graph networks. In International Conference on Learning Representations, 2018.
  59. Learning with invariances in random features and kernel models, 2021.
  60. Weisfeiler and leman go neural: Higher-order graph neural networks. Association for the Advancement of Artificial Intelligence, 2019.
  61. Max Planck. On the law of the energy distribution in the normal spectrum. Ann. Phys, 4(553):1–11, 1901.
  62. Vasco Portilheiro. A tradeoff between universality of equivariant models and learnability of symmetries. arXiv preprint arXiv:2210.09444, 2022.
  63. Self-organization of vegetation in arid ecosystems. The American Naturalist, 160(4):524–530, 2002.
  64. Loop quantum gravity and the meaning of diffeomorphism invariance. In Towards quantum gravity, pages 277–324. Springer, 2000.
  65. Stephan Rudolph et al. On the context of dimensional analysis in artificial intelligence. In International Workshop on Similarity Methods. Citeseer, 1998.
  66. Hamiltonian graph networks with ODE integrators, 2019.
  67. Erwin Schrödinger. An undulatory theory of the mechanics of atoms and molecules. Physical review, 28(6):1049, 1926.
  68. Igor R Shafarevich. Basic Algebraic Geometry 1. Springer-Verlag Berlin/Heidelberg, second edition, 1994.
  69. Data augmentation as feature manipulation: A story of desert cows and grass cows. arXiv:2203.01572, 2022.
  70. Richard P. Stanley. Smith normal form in combinatorics. Journal of Combinatorial Theory, Series A, 144:476–495, 2016.
  71. Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds. arXiv:1802.08219, 2018.
  72. Guide for the Use of the International System of Units (SI); Natl. Inst. Stand. Technol. Spec. Publ. 811, 2008 ed. National Institute of Standards and Technology, 2008.
  73. Painting with baryons: Augmenting N-body simulations with gas using deep generative models. Monthly Notices of the Royal Astronomical Society: Letters, 487(1):L24–L29, 2019.
  74. The art of data augmentation. Journal of Computational and Graphical Statistics, 10(1):1–50, 2001.
  75. Scalars are universal: Equivariant machine learning, structured like classical physics. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021.
  76. Certified robustness of graph neural networks against adversarial structural perturbation. arXiv:2008.10715, 2020a.
  77. Towards physics-informed deep learning for turbulent flow prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1457–1466, 2020b.
  78. General e(2)-equivariant steerable cnns. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32, 2019.
  79. 3d steerable cnns: Learning rotationally equivariant features in volumetric data, 2018.
  80. Understanding data augmentation for classification: when to warp? In 2016 international conference on digital image computing: techniques and applications (DICTA), pages 1–6. IEEE, 2016.
  81. How powerful are graph neural networks? arXiv:1810.00826, 2018.
  82. A simple equivariant machine learning method for dynamics based on scalars. arXiv:2110.03761, 2021.
  83. Physics-guided ai for large-scale spatiotemporal data. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD ’21, page 4088–4089, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383325. doi: 10.1145/3447548.3470793. URL https://doi.org/10.1145/3447548.3470793.
Citations (26)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Slide Deck Streamline Icon: https://streamlinehq.com

Whiteboard

Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up