---
title: Minimum-Action Learning (MAL)
url: https://www.emergentmind.com/topics/minimum-action-learning-mal
type: topic
---

# Minimum-Action Learning (MAL)

Searching arXiv for the cited MAL-related papers to ground the article with current records.
Minimum-Action Learning (MAL) is a label used for several related but non-identical learning programs built around an action-like objective. In one strand, the objective is the classical action, optical path length, or travel time, and reinforcement learning is used to recover stationary-action trajectories in physics [2011.11891]. In another, MAL denotes an energy-constrained symbolic model-selection framework that minimizes a Triple-Action functional for identifying physical force laws from noisy data [2603.16951]. Closely related work uses “minimum action” to mean the minimum number of actions required to move between states in a Markov decision process, yielding reward-free state representations based on the Minimum Action Distance (MAD) [2312.10276; 2506.09276]. A separate variational tradition studies learning by minimizing cognitive action functionals over parameter trajectories and proves existence of minimizers together with dissipative learning dynamics [1907.02517].

## 1. Scope of the term

The available literature uses the expression “Minimum-Action Learning” for distinct constructions that share a common structural idea: a learning system is driven by an objective that encodes path economy, trajectory admissibility, or parsimonious dynamical structure. The precise meaning of “action” therefore depends on the paper.

| Formulation | Action-like quantity | Representative work |
|---|---|---|
| Physics-guided RL | Classical action / optical path length / travel time | [2011.11891] |
| Symbolic law identification | Triple-Action functional | [2603.16951] |
| Reward-free state representation | Minimum Action Distance | [2312.10276], [2506.09276] |
| Variational learning dynamics | Cognitive action | [1907.02517] |

A recurring misconception is that MAL always refers to the classical variational principle of mechanics. The 2026 symbolic-identification paper explicitly states that its “Minimum-Action” refers to minimizing a compound functional—the Triple-Action—rather than the classical integral of the Lagrangian [2603.16951]. Conversely, the MAD literature uses “minimum action” to denote minimum action counts in an MDP, not a Lagrangian action integral [2312.10276; 2506.09276]. The term is therefore best understood as a family resemblance rather than a single formalism.

## 2. Least-action reinforcement learning in optics and mechanics

In the physics-guided RL formulation, the action of a trajectory $q(t)$ over $[t_0,t_1]$ is

$$
\mathcal{S}[q] \;=\; \int_{t_0}^{t_1} L\big(q(t), \dot{q}(t), t\big)\, dt,
$$

and physical trajectories render this functional stationary, with stationarity characterized by the Euler–Lagrange equation

$$
\frac{d}{dt}\Bigg(\frac{\partial L}{\partial \dot{q}}\Bigg) \;-\; \frac{\partial L}{\partial q} \;=\; 0.
$$

The MAL construction in this setting uses an RL reward that is a monotone transform of minus the action, so that maximizing expected return converges to a trajectory that minimizes the action [2011.11891].

The optics instantiation adopts Fermat’s principle. If light propagates through a medium with spatially varying refractive index $n(\mathbf{x})$, then the optical path length along a curve $\Gamma$ is

$$
\mathrm{OPL} \;=\; \int_{\Gamma} n(\mathbf{x})\, ds,
$$

and the travel time is

$$
T \;=\; \frac{1}{c}\int_{\Gamma} n(\mathbf{x})\, ds.
$$

Fermat’s principle states that actual rays satisfy

$$
\delta \int_{\Gamma} n\, ds \;=\; 0.
$$

For piecewise constant media this recovers Snell’s law at interfaces,

$$
n_1 \sin \theta_1 \;=\; n_2 \sin \theta_2.
$$

The concrete environment in [2011.11891] is a 2D grid discretization with three $50\times 50$ slabs—air, water, and glass—concatenated horizontally to form a $50\times 150$ grid. The start and goal are fixed at $A=(0,0)$ and $B=(150,50)$, with vertical interfaces at $x=50$ and $x=100$. Instead of propagating a photon cell by cell, the agent searches over two interface hit-points, using the state $s=(y_1,y_2)$, actions $\{y_1\uparrow, y_1\downarrow, y_2\uparrow, y_2\downarrow\}$, and deterministic clipped transitions. For a given state, the path is the three-segment polyline $A\to I_1\to I_2\to B$, with total travel time

$$
T(s) \;=\; n_1 \ell_1 \;+\; n_2 \ell_2 \;+\; n_3 \ell_3.
$$

Reward shaping uses the Euclidean path-integral-style score

$$
R_s(s) \;=\; N\, e^{-T(s)},
$$

and the per-round reward is the improvement over the best score found so far in the current episode. Q-learning is then applied with $\epsilon$-greedy exploration, $\epsilon=0.9$, learning rate $\alpha=10^{-3}$, discount factor $\gamma=0.9$, and fixed-horizon episodes of $300$ rounds starting from $s_{\mathrm{ini}}=(0,0)$ unless otherwise specified [2011.11891].

The principal experiment uses $n_{\mathrm{air}}=1$, $n_{\mathrm{water}}=1.3$, and $n_{\mathrm{glass}}=1.6$. The theoretical least-time path corresponds to $s_{\mathrm{theo}}=(21,37)$. Training for $100$ episodes of $300$ rounds recovers interface coordinates within one grid unit, and the learned segment angles satisfy Snell’s law within grid discretization error. The same framework also succeeds when refractive indices are altered, for example $n=(3,1,2)$ with $s_{\mathrm{ini}}=(50,50)$, and the paper reports that $100\times 300=30{,}000$ action selections suffice to reach the optimal interface coordinates robustly in this small state space [2011.11891].

The paper also draws an explicit analogy to the Euclidean path integral. By using $R_s=N e^{-T(s)}$, the RL objective mimics the Euclidean weighting $e^{-\mathcal{S}_E/\hbar}$, so that lower-action paths receive higher weight. This suggests a semiclassical interpretation: maximizing expected return drives the agent toward the stationary-action trajectory in a way that parallels dominance of stationary-phase or minimal Euclidean action paths in the propagator [2011.11891].

## 3. Energy-constrained symbolic law identification

The 2026 MAL framework addresses a different problem: identifying symbolic force laws from noisy observational data. Here MAL is defined as an energy-constrained symbolic model-selection framework that operates within a pre-specified library of interpretable basis functions and trains a small differentiable model to minimize a Triple-Action functional balancing trajectory reconstruction, architectural sparsity, and symmetry-driven energy conservation [2603.16951].

The method is motivated by the instability of finite-difference acceleration estimates under noise. Its wide-stencil acceleration-matching preprocessor reduces noise variance by $10{,}000\times$, transforming an inverse-dynamics problem with SNR $\approx 0.02$ into one with SNR $\approx 1.6$ at stride $s=10$; this preprocessing is described as the critical enabler shared by all methods tested, including SINDy variants [2603.16951].

The model class is restricted to central forces. The basis library is
$\phi_i(r)\in \{r^{-2}, r^{-1}, r, 1, r^{-3}\}$, and the force law is parameterized as

$$
\mathbf{F}(\mathbf{r}; \theta, A) \;=\; -\left[\sum_{i=1}^{K} A_i \,\theta_i\, \phi_i(r)\right] \frac{\mathbf{r}}{r},
\quad K=5,
$$

with learnable coefficients $\theta_i$ and softmax gates $A_i=\mathrm{softmax}(\ell_i/\tau)$ produced from logits $\ell_i$ and temperature $\tau$ [2603.16951]. The soft-to-discrete sharpening of these gates yields near-one-hot basis selection.

The Triple-Action objective is

$$
\mathcal{L} \;=\; \alpha_I \,\mathcal{L}_{I_{\max}} \;+\; \alpha_E \,\mathcal{L}_{E_{\min}} \;+\; \alpha_S \,\mathcal{L}_{\mathrm{Symmetry}}.
$$

Its information term combines trajectory and acceleration losses, its energy-minimization term combines coefficient sparsity and architecture entropy, and its symmetry term penalizes energy variance,

$$
\mathcal{L}_{\mathrm{Symmetry}} \;=\; \mathrm{Var}\Big[E(\mathbf{r}_k,\mathbf{v}_k)\Big],
\qquad
E(t)=T(t)+V(r(t)).
$$

For central forces with learned magnitude $f(r)=\sum_i A_i\theta_i\phi_i(r)$, the potential is taken as $V(r)=-\int^r f(\rho)\,d\rho$ up to a constant [2603.16951].

The training protocol uses synthetic 2D Kepler and Hooke benchmarks, a velocity-Verlet integrator with $\Delta t_{\mathrm{model}}=0.01$, observation interval $\Delta t_{\mathrm{obs}}=0.05$, random initialization of logits and coefficients, wide-stencil preprocessing at stride $s=10$, and a two-phase BGNO schedule: a warmup phase over epochs $1$–$50$ with $\alpha_I=1.0$, $\alpha_E=0.01$, $\tau=1.0$, followed by a sparsification phase over epochs $51$–$200$ in which $\alpha_E$ is ramped linearly from $0.01$ to $1.0$ and $\tau$ is decayed exponentially from $1.0$ to $0.05$. Optimization uses Adam with learning rate $10^{-3}$, batch size $4$, and $200$ epochs on an RTX 2080 Ti GPU [2603.16951].

The reported results separate raw basis selection from pipeline-level identification. On Kepler gravity, the gates crystallize to the $r^{-2}$ basis in $4/10$ seeds directly, the recovered coefficient is $\hat{GM}\approx 0.936$ with $6.4\%$ error, and rollouts satisfy Kepler’s third law with exponent $p=3.01\pm 0.01$. Selecting across seeds by minimum long-horizon Hamiltonian variance yields $100\%$ correct pipeline-level identification; with physics-informed gate initialization bias, direct selection also reaches $10/10$ [2603.16951]. On Hooke’s law, $9/10$ seeds directly select the $r$ basis, $\hat{k}=0.980\pm 0.001$ with $2\%$ error, and the energy diagnostic again yields $10/10$ pipeline-level identification.

The key decision criterion is long-horizon energy conservation. Over rollouts of five orbital periods, the Hamiltonian variance

$$
\sigma_H^2 \;=\; \left\langle \big(H - \langle H\rangle\big)^2 \right\rangle
$$

is substantially lower for the correct basis. On Kepler, $r^{-2}$ models conserve energy $3\times$ better than $r^{-1}$ and $6\times$ better than $r^{-3}$ despite similar short-term trajectory errors [2603.16951]. This is central to the framework’s claim that wrong laws can fit short trajectories while failing as long-horizon dynamical explanations.

The method occupies a specific niche in comparison to alternatives. With the same wide-stencil preprocessing, vanilla SINDy and ensemble-SINDy select $r^{-2}$ in $10/10$ seeds and GP-SINDy in $8/10$, at orders-of-magnitude lower compute; HNNs achieve excellent energy conservation but learn black-box Hamiltonians; LNNs fail on noisy data due to Hessian singularities [2603.16951]. The MAL paper therefore positions itself not as a universal replacement for system identification, but as an interpretable, energy-constrained model-selection procedure with explicit rollout validation.

## 4. Minimum Action Distance in reward-free representation learning

A different use of “minimum action” appears in representation learning for reward-free MDPs. In this literature, the Minimum Action Distance (MAD) is the minimum number of decision steps needed to transition between states, and learning MAD is presented as a self-supervised route to state representations for planning, goal-conditioned control, and reward shaping [2312.10276; 2506.09276].

For an MDP $\mathcal{M}=\langle \mathcal{S}, \mathcal{A}, r, P\rangle$, the one-step reachability relation is

$$
R \subseteq \mathcal{S}^2,\quad (s,s')\in R \iff \exists a\in\mathcal{A}\text{ such that }P(s' \mid s,a) >0.
$$

In the 2025 formulation, MAD is the unique solution of a linear program that maximizes the sum of pairwise distances subject to identity, one-step upper bounds, and triangle inequalities; in deterministic dynamics it coincides with shortest-path distance on the directed support graph, and in stochastic dynamics it remains defined on support only, not as an expected hitting time [2506.09276]. The 2023 formulation states the same object as the minimum number of decision steps on the directed reachability graph and emphasizes that it is generally asymmetric, since reachability can depend on direction [2312.10276].

Because enumerating the full state space is often impossible, both papers use trajectory-based supervision. For a trajectory $\tau=\{s_0,\dots,s_n\}$, the Trajectory Distance is defined by $d_{\mathrm{TD}}(s_i,s_j\mid \tau)=j-i$ for $i\le j$, and this provides an observable upper bound $d_{\mathrm{MAD}}(s_i,s_j)\le d_{\mathrm{TD}}(s_i,s_j\mid\tau)$ [2312.10276]. The 2025 paper similarly uses the time-index gap $j-i$ as supervision and builds objectives that enforce known short-horizon upper bounds while propagating multistep structure by bootstrapping [2506.09276].

The central technical issue is asymmetry. Symmetric latent norms collapse directionality and approximate $\min\{d_{\mathrm{MAD}}(s_i,s_j),d_{\mathrm{MAD}}(s_j,s_i)\}$ rather than the true directed distance [2312.10276]. To address this, the 2023 paper adopts an asymmetric seminorm based on Wide Norms:

$$
d_\phi(s_i,s_j)
=
\left\|
W\big[
\operatorname{relu}(\phi_\theta(s_i)-\phi_\theta(s_j));
\operatorname{relu}(\phi_\theta(s_j)-\phi_\theta(s_i))
\big]
\right\|_2.
$$

This construction is nonnegative, positively homogeneous, and subadditive, but generally asymmetric [2312.10276]. The 2025 paper studies several asymmetric quasimetrics, including a simple ReLU quasimetric, Wide Norm, and Interval Quasimetric Embedding (IQE), all used to define latent directed distances $d_\theta(s,t)=d_q(\phi_\theta(s),\phi_\theta(t))$ [2506.09276].

The training objectives differ in detail but share the same structure. The 2023 method minimizes squared error to trajectory-derived distances plus a hinge penalty enforcing one-step upper bounds, so that any observed transition must have learned distance at most $1$ [2312.10276]. The 2025 paper introduces MadDist and TDMadDist. MadDist uses a ratio loss, a contrastive separation term, and an upper-bound constraint term; TDMadDist adds a target encoder and a temporal-difference-style recursion of the form “one step plus the remaining distance,” with an EMA update for the target parameters [2506.09276].

These representations are used in downstream control. The 2023 paper learns a latent transition model $\rho_\zeta$ and then applies greedy goal-conditioned control,

$$
\pi(s,g)=\arg\min_{a\in A}\|\rho_\zeta(\phi_\theta(s),a)-\phi_\theta(g)\|_{\mathrm{WN}},
$$

while also noting that the learned directed distance can serve as a heuristic in model-based planning [2312.10276]. The 2025 paper emphasizes reward shaping and goal-conditioned RL, including dense rewards of the form $r_t^{\mathrm{dense}}=-d_\theta(s_t,g)$ and potential-based shaping with $\Phi_g(s)=-d_\theta(s,g)$ [2506.09276].

Empirically, the 2023 paper reports that WideNorm performs comparably to symmetric norms in symmetric environments and surpasses them in asymmetric environments, both for distance approximation and planning [2312.10276]. The 2025 paper evaluates on NoisyGridWorld, KeyDoorGridWorld, CliffWalking, and PointMaze, using Pearson correlation, Spearman correlation, and ratio coefficient of variation. It reports that MadDist and TDMadDist outperform QRL and Hilbert representations across all environments, with especially large gains in asymmetric domains and larger state spaces, and that the simple ReLU quasimetric outperforms Wide Norm and IQE in its ablations [2506.09276].

## 5. Cognitive action and variational learning dynamics

A further line of work studies learning itself as a variational problem over parameter trajectories. The 2019 paper develops cognitive action functionals on paths $q:[0,T]\to\mathbb{R}^n$, with admissible trajectories in $H^2((0,T);\mathbb{R}^n)$ and fixed initial conditions $q(0)=q^0$, $\dot q(0)=q^1$ [1907.02517]. This formulation does not use MAL terminology in the title, but its details explicitly align it with action-based learning.

A central functional is the weighted action

$$
\mathsf{W}_\varepsilon(q)
=
\int_0^T e^{-t/\varepsilon}
\Bigl(
\frac{\varepsilon^2 m}{2}|\ddot q(t)|^2 + V(q(t))
\Bigr)\,dt,
$$

together with the dissipative variant

$$
\overline{\mathsf{W}_\varepsilon}(q)
=
\int_0^T e^{-t/\varepsilon}
\Bigl(
\frac{\varepsilon^2 m}{2}|\ddot q(t)|^2
+
\frac{\varepsilon}{2}|\dot q(t)|^2
+
V(q(t))
\Bigr)\,dt.
$$

The paper also introduces a more general cognitive action $\Gamma(q)$ with time-dependent weight $\varpi(t)$, quadratic penalties on $q$, $\dot q$, and $\ddot q$, a mixed term $\gamma\,\dot q\cdot\ddot q$, and a potential $U(q,u(t))$ driven by an input stream $u(t)$ [1907.02517].

For $\mathsf{W}_\varepsilon$, stationarity yields a fourth-order Euler–Lagrange equation,

$$
\varepsilon^2 m\, q^{(4)}(t)\;-\;2\varepsilon m\, q^{(3)}(t)\;+\; m\,\ddot q(t)\;+\;\nabla V\bigl(q(t)\bigr) \;=\; 0,
$$

with natural boundary conditions at $t=T$,

$$
\ddot q(T)=0,\qquad q^{(3)}(T)=0.
$$

This gives a fourth-order boundary-value problem for each $\varepsilon>0$ [1907.02517]. The paper emphasizes that, unlike classical Hamiltonian action where stationarity may occur at saddle points, its cognitive action functionals are coercive and lower semicontinuous under stated assumptions, so genuine minima exist.

The existence theorem concerns the generalized $\Gamma$ functional under assumptions that $T\in(0,\infty)$, $\varpi\in L^\infty(0,T)$ with positive lower and finite upper bounds, $U$ is continuous and bounded below, $u(t)$ is continuous, and the coefficients satisfy $\alpha,\beta,\kappa>0$ [1907.02517]. Under these conditions, the minimization problem over the admissible set has a solution. The proof uses the direct method of the calculus of variations, with coercivity from the quadratic terms and lower semicontinuity under strong $H^1$ convergence and weak $L^2$ convergence of second derivatives.

The asymptotic limit reconnects this framework to classical optimization. As $\varepsilon\to 0$, minimizers of $\mathsf{W}_\varepsilon$ converge weakly in $H^1$ to the unique solution of the Newton initial-value problem. For the dissipative variant, the limit dynamics become

$$
m \ddot q(t) + \delta \dot q(t) + \nabla V(q(t)) = 0,
$$

which is the heavy-ball equation, and the further small-mass limit gives gradient flow,

$$
\dot q(t) = -\frac{1}{\delta}\nabla V(q(t)).
$$

The paper also derives energy dissipation for the autonomous dissipative system through

$$
\frac{d}{dt}E(t) = -\delta |\dot q(t)|^2 \le 0,
$$

where $E(t)=\frac{m}{2}|\dot q(t)|^2+V(q(t))$ [1907.02517]. No experiments are reported; the contribution is theoretical, establishing well-posedness and a variational route from higher-order action minimization to causal dissipative learning dynamics.

## 6. Relations, misconceptions, and open problems

The main conceptual relation across these literatures is structural rather than definitional. Each framework trains by optimizing an action-like quantity over trajectories, models, or latent distances. The objects being optimized, however, are different: optical travel time in physics-guided RL [2011.11891], a Triple-Action combining information, sparsity, and symmetry in symbolic law identification [2603.16951], minimum step count in reward-free MDPs [2312.10276; 2506.09276], and cognitive action over parameter trajectories in variational learning theory [1907.02517]. This suggests that “MAL” is best read as a family of action-centric learning principles rather than a single standardized method.

Several limitations are formulation-specific. In the optics RL setting, the method requires only the ability to evaluate path action and does not require analytic gradients or model equations, but it is sample-inefficient relative to closed-form physics, sensitive to reward shaping, biased by discretization, and does not scale naturally to high-dimensional continuous path search without function approximation [2011.11891]. In the symbolic-identification setting, the library is fixed, near-confounders such as $\{r^{-2.5},r^{-1.5}\}$ can reduce raw correct selection to about $20\%$, and without wide-stencil variance reduction all tested methods fail [2603.16951]. In the MAD literature, performance depends on trajectory coverage, stochastic dynamics are treated through support rather than transition probabilities, and asymmetric seminorms or quasimetrics can still violate identity of indiscernibles if the encoder collapses distinct states [2312.10276; 2506.09276]. In the cognitive-action framework, existence is established but uniqueness and explicit Euler–Lagrange equations for the full $\Gamma$ functional are not provided [1907.02517].

The literature also clarifies several common misunderstandings. MAD is not an expected hitting time; in stochastic environments it is defined on the support graph and is invariant to transition probabilities as long as support does not change [2506.09276]. The symbolic MAL framework is not open-ended scientific discovery; it performs model selection among provided candidates in a pre-specified basis library [2603.16951]. The least-action RL paper does not simulate photon propagation cell by cell in its main experiment; it searches over discrete interface crossing points [2011.11891].

Future directions stated in the papers remain diverse. The physics-guided RL work points to more complex environments and deep RL, together with further development of the path-integral analogy [2011.11891]. The symbolic-identification framework proposes open-ended symbolic regression, dimensional analysis and gauge constraints, extensions to non-conservative systems, multi-body dynamics, and real observational datasets [2603.16951]. The MAD papers point toward stronger bounds, hybrid planning-learning integration, expected-step variants for stochastic control, high-dimensional observations, and partial observability with sequence encoders [2312.10276; 2506.09276]. The cognitive-action framework suggests a principled basis for temporally coherent and regularized learning dynamics that connect least-action ideas to heavy-ball and gradient-flow optimization [1907.02517].

Across these strands, the enduring theme is that learning can be organized around objects that encode admissible paths, economical propagation, or parsimonious dynamical structure. What changes from one MAL formulation to another is the meaning of the “action,” the geometry it imposes, and the kind of inference problem it is meant to solve.

Source: https://www.emergentmind.com/topics/minimum-action-learning-mal