Compositional Dynamics in Learning and Mechanics

Abstract: We give a single compositional setting in which gradient-based learning and Hamiltonian-style mechanics appear as functorial semantics. The syntax is an operad Arr whose objects are input-output interfaces (pairs of manifolds) and whose morphisms are smooth adaptive arrangements, which consist of a reactive parameter space, a lens given by smooth output and input maps, and a real-valued potential. The main technical result of the paper is what we call lens internalization, a lax symmetric monoidal functor Lens(C) $\to$ C associated to any symmetric monoidal closed category C. Using it, we provide two functors $Φ<em>\text{phase}$, $Φ</em>\text{conf}$: Arr $\to$ PC into the 2-category of polynomial coalgebras -- input-output discrete dynamical systems -- which we take as the semantics category. $Φ<em>\text{phase}$ stores both position and momentum, whereas $Φ</em>\text{conf}$ stores only position. When applied to a parameterized function, $Φ<em>\text{conf}$ recovers the gradient descent training algorithm, with backpropagation as the lens' backward pass. When applied to harmonic particles wired together -- in series, or according to any finite directed graph -- one diagram yields two different regimes, both of which are governed by the graph Laplacian: $Φ</em>\text{phase}$ gives the discrete wave equation, which is conservative and second-order, and $Φ_\text{conf}$ gives the discrete heat equation, which is dissipative and first-order. They are two semantics of one adaptive arrangement, e.g. with the same potential in each case. And because Arr is an operad, such diagrams nest -- larger systems wired from smaller ones -- and each semantics assembles a system's dynamics functorially from its parts. These dynamics are moreover executable: a parameterized neural network and a graph of particles both compile, by the same construction, to explicit state machines one can run.

• The paper introduces an operadic framework of smooth adaptive arrangements that unifies gradient descent learning with Hamiltonian mechanics through categorical structures.
• It leverages functorial semantics to map adaptive arrangements to discrete dynamical systems, recovering classical algorithms such as backpropagation and Newton’s method.
• The approach provides modular, executable simulation pipelines for both learning algorithms and physical systems, advancing automatic differentiation and innovative dynamical simulations.

Authoritative Summary of "Compositional Dynamics in Learning and Mechanics" (2606.28984)

Overview and Central Constructions

"Compositional Dynamics in Learning and Mechanics" (2606.28984) presents a categorical and operadic framework unifying the structural aspects of gradient-based learning (e.g., gradient descent in neural networks) and Hamiltonian mechanics. The core contribution is the formalization of an operad $\sarr$ of smooth adaptive arrangements, whose morphisms encapsulate parametric, potentially adaptive, couplings between subsystems, and encompass both the architectures of learning algorithms and physical mechanical systems.

This syntax is mapped functorially into a semantics given by the monoidal 2-category $\pc$ of polynomial coalgebras—interpreted as input-output discrete dynamical systems. The categorical machinery leverages symmetric monoidal closed categories, lens structures, and the recent advances in polynomial functor calculus and dynamical system coalgebra.

One of the principal technical results is the lens internalization functor $\Theta: {\rm Lens}_C \rightarrow C$, a lax symmetric monoidal functor which enables seamless translation from bidirectional lens-based system interfaces to the category of states or polynomials understood as system interfaces.

The Syntax Operad: Smooth Adaptive Arrangements

The objects of the operad $\sarr$ are input-output interfaces, specifically pairs of manifolds, representing inputs and outputs of system components (or, in ML, module or layer interfaces). Its morphisms are smooth adaptive arrangements, parameterized by:

• A reactive parameter space $Q$ (a finite-dimensional manifold with additional structure ensuring a map from covectors to vectors—see below)
• A lens: consisting of a smooth output map $+f: Q \times +M \to +N$ and a smooth input map $f: Q \times +M \times N \to M$, encoding bidirectional data flow
• A real-valued potential $U: Q \times +M \times N \to \mathbb{R}$, which drives the system's evolution

These "adaptive arrangements" generalize classical wiring diagrams by allowing not only static composition but also adaptive and dynamic (parameter-dependent) coupling, dictated by potentials.

Functorial Semantics and Dynamical Systems

The semantic passage from syntax (adaptive arrangements) to dynamics is executed through two main functors $\Phiphase$ and $\Phiconf$:

• $\pc$0 (Phase Integrator): Interprets arrangements as phase space systems, tracking both positions and momenta, yielding a discrete-time, conservative, second-order evolution (analogous to Hamiltonian/finite-difference wave equations)
• $\pc$1 (Configuration Integrator): Interprets arrangements as purely configuration-space dynamics, tracking only positions, yielding a dissipative, first-order evolution (analogous to gradient flow/heat equations)

Both functors descend from a uniform categorical construction utilizing the Para functor: parameterizing the system arrangements over a symmetric monoidal parameter category (reactive vector spaces), and then composing with lens internalization to land in the coalgebraic world.

Categorical Details and Internalization

• Lens Internalization: For any symmetric monoidal closed category $\pc$2, lens interfaces are internalized as objects $\pc$3, where $\pc$4 denotes the internal hom and $\pc$5 represents the output side; this internalization is central for transforming bidirectional, compositional system interfaces into executable polynomial interfaces.
• Cotangent Functor: This construction leverages the cotangent functor from manifolds to polynomials, sending $\pc$6 to the polynomial encoding its cotangent fibers, thus functorially relating the differential structure of manifolds to the coalgebraic semantics.

Key Results and Examples

Recovery of Classical Algorithms

• Gradient Descent and Backpropagation: Under $\pc$7, the framework recovers the standard gradient descent update rules for parameterized smooth maps (such as neural networks). The backward map of the lens structure is identified with the backpropagation algorithm, showing that reverse-mode automatic differentiation is categorically a lens backward pass.
• Newton's Method: The configuration integrator generalizes Newton's optimization algorithm, treating the second derivative (Hessian) as a reactive structure.

Graph and Physical Systems

A significant example is the treatment of harmonic particles coupled via network topologies:

• Discrete Wave and Heat Equations: When parameterized by a finite directed graph, $\pc$8 yields conservative discrete wave equations (second-order), while $\pc$9 yields dissipative heat equations (first-order), both governed by the graph Laplacian. The same arrangement with identical potential yields both regimes; the only distinction lies in the "integrator" chosen.

Conservation Laws

• The phase integrator preserves the canonical symplectic pairing exactly, for closed harmonic systems with constant symmetric reaction and quadratic potential—discrete symplecticity is categorically established.

Numerical and Structural Claims

• The mapping of architecture and parameters to explicit discrete state machines is constructive: the categorical compilation is executable and yields algorithms with numerical behavior matching classical paradigms. E.g., training a neural network via backpropagation is realized as a parameter update in a compositional dynamical system.
• The functorial construction ensures compositionality at every level: all system dynamics, whether in learning or mechanics, assemble faithfully under composition of arrangements, mirroring the modularity of both physical systems and machine learning architectures.

Theoretical and Practical Implications

Theoretical

• This work rigorously unifies physical and machine learning dynamics in a common algebraic-compositional framework, giving a precise sense in which systems as diverse as neural networks and Hamiltonian mechanics are governed by the same underlying syntactic and categorical principles.
• The lens internalization and use of polynomial coalgebras bridge the gap between bidirectional systems, compositional open-system approaches, and executable dynamical system semantics.

Practical

• The categorical translation provides an implementation-ready pathway: arrangements (such as neural networks or mechanical networks) compile to concrete dynamical system state machines, opening avenues for correct-by-construction, modular simulation and differentiation tools, potentially extending to automatic differentiation, symbolic computation, and learned physics engines.
• The explicit treatment of adaptive coupling (e.g., adaptive spring materials, learning rates or regularizers) formalizes and generalizes key ML and physics design principles.

Future Directions

• Extension from vector space to general manifold parameter spaces (reactive manifolds with connection) is anticipated to broaden applicability (see generalization notes).
• The framework supports extensions to multi-stage integrators and other numerical discretizations beyond Euler, allowing study of advanced symplectic integration and their categorical semantics.
• The "linear stratum" (quadratic potentials, linear lens maps, constant sharp) is posited as a core of compositional, symplectic port-Hamiltonian systems, with open diagrams composing to Lagrangian relations—this direction is suggested for future work.

Conclusion

This paper rigorously develops a compositional, functorial semantics for discrete dynamical systems unifying gradient-based learning and Hamiltonian mechanics, grounded in operad theory and categorical coalgebra. The explicit translation between learning and physics is substantiated by concrete examples (gradient descent, Newton's method, discrete wave and heat equations on graphs), categorical symplecticity results, and an executable compilation pipeline. The compositional approach affords both practical benefits—modular, correctly-interpreted simulation and learning systems—and theoretical unification, with potential for further expansion into non-linear, symplectic, and multi-stage system semantics.