Noether's razor: Learning Conserved Quantities (2410.08087v1)

Abstract: Symmetries have proven useful in machine learning models, improving generalisation and overall performance. At the same time, recent advancements in learning dynamical systems rely on modelling the underlying Hamiltonian to guarantee the conservation of energy. These approaches can be connected via a seminal result in mathematical physics: Noether's theorem, which states that symmetries in a dynamical system correspond to conserved quantities. This work uses Noether's theorem to parameterise symmetries as learnable conserved quantities. We then allow conserved quantities and associated symmetries to be learned directly from train data through approximate Bayesian model selection, jointly with the regular training procedure. As training objective, we derive a variational lower bound to the marginal likelihood. The objective automatically embodies an Occam's Razor effect that avoids collapse of conservation laws to the trivial constant, without the need to manually add and tune additional regularisers. We demonstrate a proof-of-principle on $n$-harmonic oscillators and $n$-body systems. We find that our method correctly identifies the correct conserved quantities and U($n$) and SE($n$) symmetry groups, improving overall performance and predictive accuracy on test data.

• The paper presents a method to parameterize symmetries as learnable conserved quantities using Noether’s theorem.
• It employs approximate Bayesian model selection with a variational lower bound to prevent trivial solutions during training.
• Experiments on harmonic oscillators and n-body systems show improved prediction accuracy and automatic detection of symmetry groups.

An Overview of "Noether's Razor: Learning Conserved Quantities"

The paper "Noether's Razor: Learning Conserved Quantities" presents a novel approach to incorporating symmetries in machine learning models, specifically targeting Hamiltonian dynamics. The authors employ Noether's theorem, a pivotal result in physics that connects symmetries to conserved quantities, to introduce an innovative framework for learning these conserved quantities directly from training data. This work represents a step forward in the field of physics-informed machine learning by enabling the automatic discovery of symmetries that improve model generalization and performance without requiring explicit pre-imposed constraints or extensive hyperparameter tuning.

Key Contributions

1. Symmetry Parameterization via Noether's Theorem: The authors propose a method to parameterize symmetries as conserved quantities. By leveraging Noether's theorem, they can map symmetries present in dynamical systems to specific conservations, crucially allowing these quantities to be learnable within model frameworks.
2. Bayesian Model Selection: The paper employs approximate Bayesian model selection to integrate the learning of conserved quantities with the standard training of machine learning models. A variational lower bound serves as the objective for this training, embodying an implicit Occam's Razor effect that discourages trivial solutions.
3. Proof-of-Concept on Dynamical Systems: Through experiments on $n$-harmonic oscillators and $n$-body systems, the method demonstrates its capacity to automatically discern the correct conserved quantities and symmetry groups, such as U($n$) and SE($n$), achieving improved predictive accuracy.

Experimental Results

The experimental results bolster the authors' assertions:

• For the simple harmonic oscillator, models employing learned symmetries matched the performance of models with fixed, predefined symmetries, significantly outperforming those without these symmetries.
• In the case of $n-$harmonic oscillators, the method successfully identified the unitary group symmetries (U($n$)), maintaining accurate predictions over extended periods.
• For $n$-body problems, the Noether's Razor approach identified symmetries within complex gravitational interactions, enhancing both generalization and prediction accuracy compared to conventional models.

Implications and Future Directions

The proposed methodology opens new avenues for incorporating physical insights into machine learning models, especially those modeling dynamical systems. The integration of GPs through the use of deep Hamiltonian neural networks reflects substantial progress in balancing computational tractability with model expressiveness. Future work may explore richer classes of conserved quantities beyond quadratic forms, potentially incorporating neural networks to represent them. Such exploration requires careful regularization to prevent overfitting due to increased complexity.

Additionally, the authors pave the way for applying this approach to non-integrable systems and real-world datasets where conservation laws provide a valuable inductive bias. There is also the prospect of applying similar techniques to other domains where symmetry plays a crucial role, potentially including quantum mechanics or large-scale environmental models.

By efficiently utilizing Bayesian model selection, "Noether's Razor" stands out by merging foundational physics with advanced machine learning techniques, postulating an efficient framework for physically-consistent learning models. This aligns with broader AI research efforts aimed at integrating domain knowledge seamlessly into data-driven models, fostering systems that are both interpretable and performant.