Minimum-Action Learning (MAL)
- Minimum-Action Learning (MAL) is a family of methods that optimize action-like objectives to achieve parsimonious and physically consistent trajectories.
- It integrates diverse approaches including physics-guided reinforcement learning, energy-constrained symbolic law identification, and reward-free state representation with minimum action distances.
- MAL offers practical applications in optics, variational dynamics, and system identification, driving progress in efficient control and interpretable model selection.
Searching arXiv for the cited MAL-related papers to ground the article with current records. Minimum-Action Learning (MAL) is a label used for several related but non-identical learning programs built around an action-like objective. In one strand, the objective is the classical action, optical path length, or travel time, and reinforcement learning is used to recover stationary-action trajectories in physics (Jin et al., 2020). In another, MAL denotes an energy-constrained symbolic model-selection framework that minimizes a Triple-Action functional for identifying physical force laws from noisy data (Frasch, 16 Mar 2026). Closely related work uses “minimum action” to mean the minimum number of actions required to move between states in a Markov decision process, yielding reward-free state representations based on the Minimum Action Distance (MAD) (Steccanella et al., 2023, Steccanella et al., 10 Jun 2025). A separate variational tradition studies learning by minimizing cognitive action functionals over parameter trajectories and proves existence of minimizers together with dissipative learning dynamics (Betti et al., 2019).
1. Scope of the term
The available literature uses the expression “Minimum-Action Learning” for distinct constructions that share a common structural idea: a learning system is driven by an objective that encodes path economy, trajectory admissibility, or parsimonious dynamical structure. The precise meaning of “action” therefore depends on the paper.
| Formulation | Action-like quantity | Representative work |
|---|---|---|
| Physics-guided RL | Classical action / optical path length / travel time | (Jin et al., 2020) |
| Symbolic law identification | Triple-Action functional | (Frasch, 16 Mar 2026) |
| Reward-free state representation | Minimum Action Distance | (Steccanella et al., 2023, Steccanella et al., 10 Jun 2025) |
| Variational learning dynamics | Cognitive action | (Betti et al., 2019) |
A recurring misconception is that MAL always refers to the classical variational principle of mechanics. The 2026 symbolic-identification paper explicitly states that its “Minimum-Action” refers to minimizing a compound functional—the Triple-Action—rather than the classical integral of the Lagrangian (Frasch, 16 Mar 2026). Conversely, the MAD literature uses “minimum action” to denote minimum action counts in an MDP, not a Lagrangian action integral (Steccanella et al., 2023, Steccanella et al., 10 Jun 2025). The term is therefore best understood as a family resemblance rather than a single formalism.
2. Least-action reinforcement learning in optics and mechanics
In the physics-guided RL formulation, the action of a trajectory over is
and physical trajectories render this functional stationary, with stationarity characterized by the Euler–Lagrange equation
The MAL construction in this setting uses an RL reward that is a monotone transform of minus the action, so that maximizing expected return converges to a trajectory that minimizes the action (Jin et al., 2020).
The optics instantiation adopts Fermat’s principle. If light propagates through a medium with spatially varying refractive index , then the optical path length along a curve is
and the travel time is
Fermat’s principle states that actual rays satisfy
For piecewise constant media this recovers Snell’s law at interfaces,
The concrete environment in (Jin et al., 2020) is a 2D grid discretization with three 0 slabs—air, water, and glass—concatenated horizontally to form a 1 grid. The start and goal are fixed at 2 and 3, with vertical interfaces at 4 and 5. Instead of propagating a photon cell by cell, the agent searches over two interface hit-points, using the state 6, actions 7, and deterministic clipped transitions. For a given state, the path is the three-segment polyline 8, with total travel time
9
Reward shaping uses the Euclidean path-integral-style score
0
and the per-round reward is the improvement over the best score found so far in the current episode. Q-learning is then applied with 1-greedy exploration, 2, learning rate 3, discount factor 4, and fixed-horizon episodes of 5 rounds starting from 6 unless otherwise specified (Jin et al., 2020).
The principal experiment uses 7, 8, and 9. The theoretical least-time path corresponds to 0. Training for 1 episodes of 2 rounds recovers interface coordinates within one grid unit, and the learned segment angles satisfy Snell’s law within grid discretization error. The same framework also succeeds when refractive indices are altered, for example 3 with 4, and the paper reports that 5 action selections suffice to reach the optimal interface coordinates robustly in this small state space (Jin et al., 2020).
The paper also draws an explicit analogy to the Euclidean path integral. By using 6, the RL objective mimics the Euclidean weighting 7, so that lower-action paths receive higher weight. This suggests a semiclassical interpretation: maximizing expected return drives the agent toward the stationary-action trajectory in a way that parallels dominance of stationary-phase or minimal Euclidean action paths in the propagator (Jin et al., 2020).
3. Energy-constrained symbolic law identification
The 2026 MAL framework addresses a different problem: identifying symbolic force laws from noisy observational data. Here MAL is defined as an energy-constrained symbolic model-selection framework that operates within a pre-specified library of interpretable basis functions and trains a small differentiable model to minimize a Triple-Action functional balancing trajectory reconstruction, architectural sparsity, and symmetry-driven energy conservation (Frasch, 16 Mar 2026).
The method is motivated by the instability of finite-difference acceleration estimates under noise. Its wide-stencil acceleration-matching preprocessor reduces noise variance by 8, transforming an inverse-dynamics problem with SNR 9 into one with SNR 0 at stride 1; this preprocessing is described as the critical enabler shared by all methods tested, including SINDy variants (Frasch, 16 Mar 2026).
The model class is restricted to central forces. The basis library is 2, and the force law is parameterized as
3
with learnable coefficients 4 and softmax gates 5 produced from logits 6 and temperature 7 (Frasch, 16 Mar 2026). The soft-to-discrete sharpening of these gates yields near-one-hot basis selection.
The Triple-Action objective is
8
Its information term combines trajectory and acceleration losses, its energy-minimization term combines coefficient sparsity and architecture entropy, and its symmetry term penalizes energy variance,
9
For central forces with learned magnitude 0, the potential is taken as 1 up to a constant (Frasch, 16 Mar 2026).
The training protocol uses synthetic 2D Kepler and Hooke benchmarks, a velocity-Verlet integrator with 2, observation interval 3, random initialization of logits and coefficients, wide-stencil preprocessing at stride 4, and a two-phase BGNO schedule: a warmup phase over epochs 5–6 with 7, 8, 9, followed by a sparsification phase over epochs 0–1 in which 2 is ramped linearly from 3 to 4 and 5 is decayed exponentially from 6 to 7. Optimization uses Adam with learning rate 8, batch size 9, and 0 epochs on an RTX 2080 Ti GPU (Frasch, 16 Mar 2026).
The reported results separate raw basis selection from pipeline-level identification. On Kepler gravity, the gates crystallize to the 1 basis in 2 seeds directly, the recovered coefficient is 3 with 4 error, and rollouts satisfy Kepler’s third law with exponent 5. Selecting across seeds by minimum long-horizon Hamiltonian variance yields 6 correct pipeline-level identification; with physics-informed gate initialization bias, direct selection also reaches 7 (Frasch, 16 Mar 2026). On Hooke’s law, 8 seeds directly select the 9 basis, 0 with 1 error, and the energy diagnostic again yields 2 pipeline-level identification.
The key decision criterion is long-horizon energy conservation. Over rollouts of five orbital periods, the Hamiltonian variance
3
is substantially lower for the correct basis. On Kepler, 4 models conserve energy 5 better than 6 and 7 better than 8 despite similar short-term trajectory errors (Frasch, 16 Mar 2026). This is central to the framework’s claim that wrong laws can fit short trajectories while failing as long-horizon dynamical explanations.
The method occupies a specific niche in comparison to alternatives. With the same wide-stencil preprocessing, vanilla SINDy and ensemble-SINDy select 9 in 0 seeds and GP-SINDy in 1, at orders-of-magnitude lower compute; HNNs achieve excellent energy conservation but learn black-box Hamiltonians; LNNs fail on noisy data due to Hessian singularities (Frasch, 16 Mar 2026). The MAL paper therefore positions itself not as a universal replacement for system identification, but as an interpretable, energy-constrained model-selection procedure with explicit rollout validation.
4. Minimum Action Distance in reward-free representation learning
A different use of “minimum action” appears in representation learning for reward-free MDPs. In this literature, the Minimum Action Distance (MAD) is the minimum number of decision steps needed to transition between states, and learning MAD is presented as a self-supervised route to state representations for planning, goal-conditioned control, and reward shaping (Steccanella et al., 2023, Steccanella et al., 10 Jun 2025).
For an MDP 2, the one-step reachability relation is
3
In the 2025 formulation, MAD is the unique solution of a linear program that maximizes the sum of pairwise distances subject to identity, one-step upper bounds, and triangle inequalities; in deterministic dynamics it coincides with shortest-path distance on the directed support graph, and in stochastic dynamics it remains defined on support only, not as an expected hitting time (Steccanella et al., 10 Jun 2025). The 2023 formulation states the same object as the minimum number of decision steps on the directed reachability graph and emphasizes that it is generally asymmetric, since reachability can depend on direction (Steccanella et al., 2023).
Because enumerating the full state space is often impossible, both papers use trajectory-based supervision. For a trajectory 4, the Trajectory Distance is defined by 5 for 6, and this provides an observable upper bound 7 (Steccanella et al., 2023). The 2025 paper similarly uses the time-index gap 8 as supervision and builds objectives that enforce known short-horizon upper bounds while propagating multistep structure by bootstrapping (Steccanella et al., 10 Jun 2025).
The central technical issue is asymmetry. Symmetric latent norms collapse directionality and approximate 9 rather than the true directed distance (Steccanella et al., 2023). To address this, the 2023 paper adopts an asymmetric seminorm based on Wide Norms:
00
This construction is nonnegative, positively homogeneous, and subadditive, but generally asymmetric (Steccanella et al., 2023). The 2025 paper studies several asymmetric quasimetrics, including a simple ReLU quasimetric, Wide Norm, and Interval Quasimetric Embedding (IQE), all used to define latent directed distances 01 (Steccanella et al., 10 Jun 2025).
The training objectives differ in detail but share the same structure. The 2023 method minimizes squared error to trajectory-derived distances plus a hinge penalty enforcing one-step upper bounds, so that any observed transition must have learned distance at most 02 (Steccanella et al., 2023). The 2025 paper introduces MadDist and TDMadDist. MadDist uses a ratio loss, a contrastive separation term, and an upper-bound constraint term; TDMadDist adds a target encoder and a temporal-difference-style recursion of the form “one step plus the remaining distance,” with an EMA update for the target parameters (Steccanella et al., 10 Jun 2025).
These representations are used in downstream control. The 2023 paper learns a latent transition model 03 and then applies greedy goal-conditioned control,
04
while also noting that the learned directed distance can serve as a heuristic in model-based planning (Steccanella et al., 2023). The 2025 paper emphasizes reward shaping and goal-conditioned RL, including dense rewards of the form 05 and potential-based shaping with 06 (Steccanella et al., 10 Jun 2025).
Empirically, the 2023 paper reports that WideNorm performs comparably to symmetric norms in symmetric environments and surpasses them in asymmetric environments, both for distance approximation and planning (Steccanella et al., 2023). The 2025 paper evaluates on NoisyGridWorld, KeyDoorGridWorld, CliffWalking, and PointMaze, using Pearson correlation, Spearman correlation, and ratio coefficient of variation. It reports that MadDist and TDMadDist outperform QRL and Hilbert representations across all environments, with especially large gains in asymmetric domains and larger state spaces, and that the simple ReLU quasimetric outperforms Wide Norm and IQE in its ablations (Steccanella et al., 10 Jun 2025).
5. Cognitive action and variational learning dynamics
A further line of work studies learning itself as a variational problem over parameter trajectories. The 2019 paper develops cognitive action functionals on paths 07, with admissible trajectories in 08 and fixed initial conditions 09, 10 (Betti et al., 2019). This formulation does not use MAL terminology in the title, but its details explicitly align it with action-based learning.
A central functional is the weighted action
11
together with the dissipative variant
12
The paper also introduces a more general cognitive action 13 with time-dependent weight 14, quadratic penalties on 15, 16, and 17, a mixed term 18, and a potential 19 driven by an input stream 20 (Betti et al., 2019).
For 21, stationarity yields a fourth-order Euler–Lagrange equation,
22
with natural boundary conditions at 23,
24
This gives a fourth-order boundary-value problem for each 25 (Betti et al., 2019). The paper emphasizes that, unlike classical Hamiltonian action where stationarity may occur at saddle points, its cognitive action functionals are coercive and lower semicontinuous under stated assumptions, so genuine minima exist.
The existence theorem concerns the generalized 26 functional under assumptions that 27, 28 with positive lower and finite upper bounds, 29 is continuous and bounded below, 30 is continuous, and the coefficients satisfy 31 (Betti et al., 2019). Under these conditions, the minimization problem over the admissible set has a solution. The proof uses the direct method of the calculus of variations, with coercivity from the quadratic terms and lower semicontinuity under strong 32 convergence and weak 33 convergence of second derivatives.
The asymptotic limit reconnects this framework to classical optimization. As 34, minimizers of 35 converge weakly in 36 to the unique solution of the Newton initial-value problem. For the dissipative variant, the limit dynamics become
37
which is the heavy-ball equation, and the further small-mass limit gives gradient flow,
38
The paper also derives energy dissipation for the autonomous dissipative system through
39
where 40 (Betti et al., 2019). No experiments are reported; the contribution is theoretical, establishing well-posedness and a variational route from higher-order action minimization to causal dissipative learning dynamics.
6. Relations, misconceptions, and open problems
The main conceptual relation across these literatures is structural rather than definitional. Each framework trains by optimizing an action-like quantity over trajectories, models, or latent distances. The objects being optimized, however, are different: optical travel time in physics-guided RL (Jin et al., 2020), a Triple-Action combining information, sparsity, and symmetry in symbolic law identification (Frasch, 16 Mar 2026), minimum step count in reward-free MDPs (Steccanella et al., 2023, Steccanella et al., 10 Jun 2025), and cognitive action over parameter trajectories in variational learning theory (Betti et al., 2019). This suggests that “MAL” is best read as a family of action-centric learning principles rather than a single standardized method.
Several limitations are formulation-specific. In the optics RL setting, the method requires only the ability to evaluate path action and does not require analytic gradients or model equations, but it is sample-inefficient relative to closed-form physics, sensitive to reward shaping, biased by discretization, and does not scale naturally to high-dimensional continuous path search without function approximation (Jin et al., 2020). In the symbolic-identification setting, the library is fixed, near-confounders such as 41 can reduce raw correct selection to about 42, and without wide-stencil variance reduction all tested methods fail (Frasch, 16 Mar 2026). In the MAD literature, performance depends on trajectory coverage, stochastic dynamics are treated through support rather than transition probabilities, and asymmetric seminorms or quasimetrics can still violate identity of indiscernibles if the encoder collapses distinct states (Steccanella et al., 2023, Steccanella et al., 10 Jun 2025). In the cognitive-action framework, existence is established but uniqueness and explicit Euler–Lagrange equations for the full 43 functional are not provided (Betti et al., 2019).
The literature also clarifies several common misunderstandings. MAD is not an expected hitting time; in stochastic environments it is defined on the support graph and is invariant to transition probabilities as long as support does not change (Steccanella et al., 10 Jun 2025). The symbolic MAL framework is not open-ended scientific discovery; it performs model selection among provided candidates in a pre-specified basis library (Frasch, 16 Mar 2026). The least-action RL paper does not simulate photon propagation cell by cell in its main experiment; it searches over discrete interface crossing points (Jin et al., 2020).
Future directions stated in the papers remain diverse. The physics-guided RL work points to more complex environments and deep RL, together with further development of the path-integral analogy (Jin et al., 2020). The symbolic-identification framework proposes open-ended symbolic regression, dimensional analysis and gauge constraints, extensions to non-conservative systems, multi-body dynamics, and real observational datasets (Frasch, 16 Mar 2026). The MAD papers point toward stronger bounds, hybrid planning-learning integration, expected-step variants for stochastic control, high-dimensional observations, and partial observability with sequence encoders (Steccanella et al., 2023, Steccanella et al., 10 Jun 2025). The cognitive-action framework suggests a principled basis for temporally coherent and regularized learning dynamics that connect least-action ideas to heavy-ball and gradient-flow optimization (Betti et al., 2019).
Across these strands, the enduring theme is that learning can be organized around objects that encode admissible paths, economical propagation, or parsimonious dynamical structure. What changes from one MAL formulation to another is the meaning of the “action,” the geometry it imposes, and the kind of inference problem it is meant to solve.