Automatic Symmetry Discovery with Lie Algebra Convolutional Network (2109.07103v2)

Abstract: Existing equivariant neural networks require prior knowledge of the symmetry group and discretization for continuous groups. We propose to work with Lie algebras (infinitesimal generators) instead of Lie groups. Our model, the Lie algebra convolutional network (L-conv) can automatically discover symmetries and does not require discretization of the group. We show that L-conv can serve as a building block to construct any group equivariant feedforward architecture. Both CNNs and Graph Convolutional Networks can be expressed as L-conv with appropriate groups. We discover direct connections between L-conv and physics: (1) group invariant loss generalizes field theory (2) Euler-Lagrange equation measures the robustness, and (3) equivariance leads to conservation laws and Noether current.These connections open up new avenues for designing more general equivariant networks and applying them to important problems in physical sciences

• The paper introduces L-conv, a Lie algebra-based convolutional method that automatically discovers continuous symmetries without prior group discretization.
• It demonstrates that L-conv can approximate group convolutions in architectures like CNNs and GCNs, streamlining the design of equivariant networks.
• The approach bridges machine learning and physics by linking symmetry discovery with concepts like the Euler-Lagrange equation and Noether currents.

Automatic Symmetry Discovery with Lie Algebra Convolutional Network

This paper introduces a novel approach to construct group equivariant neural networks, notably the Lie Algebra Convolutional Network (L-conv), using Lie algebras instead of discretized Lie groups. The methodology addresses the limitations of existing equivariant neural networks that require prior knowledge of symmetry groups and discretization for continuous groups. By leveraging Lie algebras, which represent infinitesimal generators of continuous groups, L-conv can automatically discover symmetries without needing group discretization, thereby simplifying the model design process for equivariant architectures.

The authors propose that L-conv serves as a fundamental building block applicable to constructing any group equivariant feedforward architecture. Common architectures such as Convolutional Neural Networks (CNNs) and Graph Convolutional Networks (GCNs) can be expressed through L-conv by appropriately choosing groups. This approach aligns with established principles connecting group convolutional layers to equivariance under group actions, showing that multilayer L-conv can approximate group convolutional layers effectively.

One of the key theoretical contributions of this work is the connection it establishes between L-conv and concepts from physics, namely:

1. Group invariant loss, drawn parallel to field theory, provides insights into robustness and symmetry in the loss landscape.
2. The Euler-Lagrange equation is identified as a measure of robustness.
3. Equivariance naturally leads to conservation laws and the emergence of Noether currents.

These insights point to intriguing intersections between machine learning and physics, opening avenues for more sophisticated equivariant network designs and applications in physical sciences. Furthermore, the theoretical backing of symmetry discovery in L-conv suggests potential in scientific machine learning, particularly for modeling in systems governed by physical laws.

In practical terms, the ability of L-conv to automatically uncover symmetries from data is evaluated through experiments, showcasing its capability in learning symmetry generators for cases like translational and rotational invariance. The paper's approach minimizes computational load by avoiding the exhaustive parametrization required by previous methods that explicitly encode irreducible representations or discretize continuous groups.

Moving forward, this research could influence developments in AI, particularly within scientific computing domains where symmetry plays a crucial role. The L-conv model shows promise as a more generalized and efficient means to incorporate equivariance in neural networks, offering improved generalization capabilities and reduced sample complexity.

The paper's implications suggest several pathways for advancing research:

• Exploration of L-conv in areas of quantum computing and complex systems, given the symmetry and conservation principles involved.
• Integration and enhancement of existing computational pipelines in physics with machine learning architectures that leverage L-conv.
• Application of L-conv to problems requiring conservation property recognition, potentially contributing to advancements in algorithmic stability and model robustness.

Overall, the paper presents a robust framework highlighting elegance and practicality in capturing the symmetries inherent in data, underscoring the bridge between abstract mathematical concepts and tangible computational applications.