Deflex: Deep Formula Discovery

• Deflex is a deep AI framework that automatically discovers mathematical laws for multiscale complex systems from high-dimensional data.
• It combines a neural energy model (Deflexformer) with a lambda calculus-based symbolic regression engine (Deflexpressor) to uncover scale-specific formulas.
• Deflex achieves up to 7-fold efficiency improvements over state-of-the-art methods, ensuring consistent and interpretable scientific discovery across scales.

to=arxiv_search.search 北京赛车女json {"query":"(Yu et al., 5 Jun 2026) Deflex symbolic regression multiscale complex systems", "max_results": 5} to=arxiv_search.search 玩彩神争霸json {"query":"symbolic regression multiscale energy-based model lambda calculus scientific discovery", "max_results": 10} to=arxiv_search.search 诺果json {"query":"AI Feynman symbolic regression arXiv", "max_results": 5} to=arxiv_search.search 娱乐开号json {"query":"PySR symbolic regression arXiv", "max_results": 5} to=arxiv_search.search 天天中彩票腾讯json {"query":"SINDy sparse identification nonlinear dynamics arXiv", "max_results": 5} Deflex, short for “Deep Formula discovery for complex systems,” is an end-to-end AI framework for automatically discovering mathematical formulas that govern multiscale complex systems directly from data. It is designed for systems with many interacting entities, multiscale behavior, and governing rules that may appear as equations, invariants, distributions, or higher-order laws. Deflex consists of two subsystems named Deflexformer and Deflexpressor, and across six representative complex systems with diverse behaviors it achieves up to 7-fold higher efficiency than the state-of-the-art methods while enabling automated multiscale discovery (Yu et al., 5 Jun 2026).

1. Scope and problem setting

Deflex addresses three core difficulties of scientific formula discovery in multiscale complex systems. The first is the volume challenge: huge numbers of interacting elements produce large data and high-dimensional relationships. The second is search-space explosion: the space of free-form symbolic expressions grows rapidly with variables and expression depth. The third is the scale gap: different scales are governed by different kinds of laws, with small-scale dynamics often deterministic and large-scale patterns often statistical. Existing sparse regression and symbolic regression methods are described as mostly single-scale in this setting (Yu et al., 5 Jun 2026).

Within this framework, a scale-specific formula is a rule that describes the system at a particular scale. At fine scale, this may be an equation or invariant such as energy conservation, Navier–Stokes, or Langevin dynamics. At coarse scale, it may be a distribution such as a Maxwell–Boltzmann law, a power law, or a Lévy-flight step distribution. A multiscale formula set is the collection of such formulas across scales, ideally consistent under coarse-graining or renormalization. Deflex is explicitly designed to discover small-scale equations and invariants, large-scale distributions, and their cross-scale relationships (Yu et al., 5 Jun 2026).

The intended domain is broad. The paper formulates observations as systems of particles, fluid cells, people, birds, or comparable interacting entities, and evaluates the method on particle systems, fluid dynamics, and mobility data. This positioning makes Deflex a method for automated scientific discovery rather than a domain-specific symbolic regressor.

2. Conceptual foundation: energy functions and multiscale laws

A central design choice in Deflex is the unified treatment of equations, invariants, and distributions through an energy-based formulation. All formulas are represented as energy functions $E(\mathbf{x})$ defining densities

$p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$

In this representation, equations and invariants are treated as distributions with narrow peaks, while stochastic laws are represented as broader peaks. This allows deterministic and statistical structure to be handled within the same formalism (Yu et al., 5 Jun 2026).

The unknown target of learning is an energy function $E^*(\mathbf{x})$ for the multiscale system. Deflexformer approximates this through a learned energy model

$$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$

Training uses the negative log-likelihood

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$

Because direct computation of $Z_\theta$ is intractable, the gradient of $\log Z_\theta$ is estimated using MCMC with Langevin sampling. The same energy-based view is then used to evaluate both equation-like objects and distribution-like objects, with RMSE used for equations and NLL used for distributions in the experiments (Yu et al., 5 Jun 2026).

This formulation also underlies the method’s multiscale claims. The paper evaluates cross-scale consistency through Earth Mover’s Distance under coarse-graining and reports the lowest EMD across scales for Deflex in water and fluid experiments, which is presented as evidence that the discovered formulas remain structurally stable under changes of scale (Yu et al., 5 Jun 2026).

3. Architecture: Deflexformer and Deflexpressor

Deflex consists of two subsystems named Deflexformer and Deflexpressor. Deflexformer is a Transformer-like deep energy model with decomposable blocks, while Deflexpressor is a lambda-calculus-based symbolic regression engine for explicit formula discovery, including higher-order formulas (Yu et al., 5 Jun 2026).

The input representation is

$\mathbf{X} = [\mathbf{S};\mathbf{V}],$

where $\mathbf{S}\in\mathbb{R}^{N\times T\times D_s}$ contains element-level states and $\mathbf{V}\in\mathbb{R}^{D_v}$ contains optional global features. A Fourier multiscale embedding maps these into element-level representations

$p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$0

and global representations

$p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$1

with $p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$2. Deflexformer is then built as a stack of $p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$3 identical blocks

$p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$4

so that the overall model is approximated by a composition

$p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$5

Each block maps $p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$6 to $p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$7 through two mechanisms. The first is a point-wise transformation network, implemented as a 4-layer MLP with widths $p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$8 and ReLU activations, for element-wise evolution. The second is multi-head self-attention with $p(\mathbf{x}) = \frac{1}{Z}\exp(-E(\mathbf{x})).$9 heads, factorized into spatial mixing over elements and temporal mixing over time. Residual paths and layer normalization are applied. The final global representation $E^*(\mathbf{x})$0 is mapped to scalar energy through an output module using inverse Fourier embedding (Yu et al., 5 Jun 2026).

Deflexpressor supplies the symbolic layer. Its program representation is lambda calculus with variables, abstraction, and application, extended with types, arrays, recursion, and a typed DSL verified by type inference based on algorithm W. This is the component that makes higher-order formulas expressible without hand-coded templates. The paper’s example for pairwise aggregation is

$E^*(\mathbf{x})$1

which computes a sum over a variable-length list $E^*(\mathbf{x})$2. In addition to rule-based generation and mutation, Deflexpressor uses a neural-guided autoregressive generator trained on valid expressions and mutation pairs, with hybrid exploration controlled by simulated annealing. It also uses aggregation priors of the form

$E^*(\mathbf{x})$3

biasing the search toward map-reduce structures common in physics and collective systems (Yu et al., 5 Jun 2026).

4. Training procedure and hierarchical symbolic regression

The training pipeline has two phases. First, Deflexpressor generates synthetic formulas and synthetic input-output pairs $E^*(\mathbf{x})$4 with $E^*(\mathbf{x})$5, and a single Deflexformer block is pre-trained on these data using

$E^*(\mathbf{x})$6

This gives the block a library of base transformations. Second, multiple copies of that pre-trained block are stacked into the full Deflexformer and post-trained on real complex-system data using the energy-based NLL objective with Langevin sampling (Yu et al., 5 Jun 2026).

After post-training, the model is run on both observation data and additional random samples generated by EBM sampling. This produces intermediate representation pairs of the form

$E^*(\mathbf{x})$7

which become the regression targets for symbolic discovery. The symbolic search is not performed on the entire network at once. Instead, the paper organizes the blocks into a binary tree. Leaves correspond to single blocks, and internal nodes correspond to contiguous ranges of blocks. Symbolic regression is first run at the leaves with a user-specified base expression set; then the expression sets learned by sibling nodes are merged upward as the base set for the parent. The result is a hierarchical symbolic regression procedure over block ranges that yields $E^*(\mathbf{x})$8 symbolic regression problems rather than one monolithic search (Yu et al., 5 Jun 2026).

Model selection is explicitly multi-objective. For a set of formulas $E^*(\mathbf{x})$9, Deflex considers both total loss and total complexity, where complexity $$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$0 is expression length. The discovery goal is a Pareto-optimal set $$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$1 satisfying

$$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$2

This formalizes the simplicity–accuracy trade-off central to symbolic regression, but in a setting where formulas may be higher-order and scale-specific rather than single closed-form equations (Yu et al., 5 Jun 2026).

5. Empirical results and representative discoveries

The reported evaluation spans six representative complex systems: rare-gas particle motion, pollen in water, cross-scale water particles near liquid–ice transition, 2D cylinder wake flow, 3D isotropic turbulence from JHTDB, and human and bird mobility. Baselines include Operon, SciMED, gplearn, PySR, DEAP, AI Feynman 2.0, and SINDy. Accuracy is evaluated with RMSE for equations and invariants and NLL for distributions; efficiency is measured by wall-clock time to reach a shared convergence threshold; multiscale consistency is evaluated with Earth Mover’s Distance across coarse-graining levels (Yu et al., 5 Jun 2026).

Several discovered laws are close to known targets. For energy conservation, the target

$$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$3

is discovered as

$$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$4

For the Maxwell–Boltzmann distribution, the target

$$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$5

is discovered as

$$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$6

For argon with Lennard–Jones interactions, the target

$$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$7

is discovered with coefficients close to the target:

$$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$8

For Newton’s viscosity law,

$$p(\mathbf{x}) = \frac{1}{Z_\theta}\exp\big(-\mathcal{E}_\theta(\mathbf{x})\big), \qquad Z_\theta = \int \exp\big(-\mathcal{E}_\theta(\mathbf{x})\big)\, d\mathbf{x}.$$9

is discovered as

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$0

For Navier–Stokes, the discovered expression preserves the full target structure up to a scale factor. The paper states that Deflex is the only method to recover the full 3D Navier–Stokes equation on JHTDB, while Operon and SINDy recover only a simplified 2D form without pressure (Yu et al., 5 Jun 2026).

The method also recovers coarse-grained stochastic laws. For Langevin dynamics, the target

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$1

is discovered as

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$2

For human Lévy-flight behavior, the target

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$3

is discovered as

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$4

and for avian mobility,

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$5

is discovered as

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$6

The human mobility power law

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$7

is discovered as

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$8

The paper also reports a new “crowd power law”

$$L_\theta = -\mathcal{L}(\mathcal{E}_\theta,\mathbf{X}) = \frac{1}{N}\sum_{i=1}^{N}\big(\mathcal{E}_\theta(\mathbf{x}_i)+\log Z_\theta\big).$$9

interpreted as a modulation of long-distance-step probability by nearby neighbors (Yu et al., 5 Jun 2026).

In aggregate, Deflex achieves similar RMSE to the best symbolic regression baselines on equations, lower NLL on distributions, and often much lower runtime on complex cases such as Navier–Stokes, Maxwell–Boltzmann with potential, vector navigation, and velocity distributions. The reported efficiency gain reaches up to 7-fold. On the AI Feynman benchmark, the paper reports RMSE $Z_\theta$0 in $Z_\theta$1s for Deflex and RMSE $Z_\theta$2 in 136s for Operon, while emphasizing that Deflex is designed for multiscale discovery rather than only classical single-scale symbolic regression (Yu et al., 5 Jun 2026).

6. Limitations, interpretation, and research significance

The paper identifies several limitations. Symbolic search remains inefficient because substantial effort is still spent on invalid or low-quality candidates. Scale interference can occur when small but physically important terms are overwhelmed by dominant low-frequency components during joint learning. The pipeline assumes pre-processed data and, in PDE settings, often assumes that derivatives are already available as features. Robustness to heavy noise or missing data is not deeply explored. Computation is non-trivial and uses GPUs, even though the method is reported as more efficient than the compared symbolic regression baselines in the tested regimes (Yu et al., 5 Jun 2026).

These constraints delimit the current meaning of “end-to-end” in Deflex. The method automates the discovery pipeline from structured observational data to symbolic laws, but it does not eliminate the need for task-specific data organization, coarse-graining choices, or feature preparation. Its strongest demonstrated capability is not unrestricted theorem induction, but the joint use of decomposable neural energy models and higher-order symbolic regression to expose formulas that differ across scales while remaining mutually consistent.

The broader significance of Deflex lies in that coupling. Deflexformer supplies learned representations in which multiscale structure becomes symbolically tractable, while Deflexpressor converts those representations into explicit formulas over variables, arrays, and aggregations. In the experiments this combination recovers classical laws such as Maxwell–Boltzmann, Navier–Stokes, and Langevin dynamics, as well as empirical mobility distributions and a new crowd power law. A plausible implication is that Deflex is best understood as an interface between representation learning, energy-based modeling, and higher-order symbolic reasoning for automated scientific discovery, rather than as a direct replacement for conventional symbolic regression (Yu et al., 5 Jun 2026).