Discovering Multiscale Deep Formulas in Complex Systems via Neural-Guided Lambda Calculus

Abstract: A fundamental problem in science is identifying underlying patterns of complex systems in the form of concise mathematical formulas. Current AI-based methods have shown strong performance in single-scale systems, yet remain limited in identifying scale-specific formulas in multiscale complex systems. We present Deflex, an end-to-end AI method to automatically extract multiscale formulas with potentially different forms, including invariants and distributions, from complex systems. Deflex consists of two subsystems named Deflexformer and Deflexpressor. Deflexpressor is a lambda-calculus symbolic regression model for higher-order formulas. Deflexformer is a decomposable deep energy model for learning unified representations across scales. Deflexpressor generates synthetic data to pre-train Deflexformer, which then guides formula discovery by decoupling multiscale latent relationships. Across six representative complex systems with diverse behaviors, Deflex achieves up to 7-fold higher efficiency than the state-of-the-art methods while enabling automated multiscale discovery. Our work could be a useful tool for scientific discovery across disciplines.

• The paper introduces Deflex, a neural-symbolic framework that discovers multiscale mathematical laws using unified energy-based modeling and lambda calculus-guided symbolic regression.
• It integrates a modular Transformer-like Deflexformer for cross-scale representation learning with a type-safe Deflexpressor to perform recursive, variable-arity formula discovery.
• The approach achieves state-of-the-art accuracy and up to 7x lower computational cost in recovering both canonical and distributional laws across diverse complex systems.

Neural-Guided Multiscale Formula Discovery in Complex Systems: An Expert Review of Deflex

Overview and Motivation

Automating the extraction of concise mathematical laws from high-dimensional, multiscale data remains a core challenge in scientific discovery, particularly in the context of complex systems where relationships span deterministic invariants and emergent stochastic distributions across scales. The paper "Discovering Multiscale Deep Formulas in Complex Systems via Neural-Guided Lambda Calculus" (2606.07426) introduces Deflex, an end-to-end neural-symbolic framework designed to address this objective. The framework’s key innovations are a unified distribution-based formalism for invariants and distributions, a decomposable energy-based neural network (Deflexformer) for cross-scale representation learning, and a lambda-calculus-based symbolic regression module (Deflexpressor) capable of expressing higher-order relationships without restrictive structural assumptions.

Deflex addresses three primary obstacles in automatic multiscale law discovery: 1) prohibitively high-dimensional search spaces due to large numbers of interacting entities, 2) combinatoric explosion of admissible formula structures, and 3) the scale gap, i.e., the frequent emergence of qualitatively different rules at different spatiotemporal resolutions.

Deflex Architecture

Deflex operationalizes formula discovery through the tight integration of two subsystems: the neural Deflexformer and the symbolic Deflexpressor. The workflow, as illustrated in (Figure 1) and described below, is sequenced to leverage synthetic data generation for neural pre-training, modular representation learning, cross-scale interpretability, and iterative symbolic regression guided by neural representations.

Figure 1: The end-to-end workflow of Deflex, demonstrating data generation, Deflexformer pre-training, cascading, inference, and hierarchical symbolic regression.

The Deflexformer is a stackable, modular Transformer-like energy-based model. The architecture supports the decomposition of learned functions into tractable components for subsequent symbolic identification. Each block processes spatiotemporal information via separated element-wise and global attention layers, enabling efficient modeling of interactions among large numbers of entities:

Figure 2: Deflexformer architecture, highlighting compositional self-attention and modular transformation at element and global levels.

Deflexpressor, utilizing an enriched, type-safe lambda calculus, supports expressive, executable formula representations encompassing aggregation, higher-order mapping, and recursion. Candidate expressions are synthesized via hybrid rule-based and neural autoregressive transformations, pruned for type-correctness, and iteratively refined through mutation and selection—substantially expanding the space of discoverable relationships compared to classic expression trees.

Methodological Innovations

Unified Distributional Representation:

Deflex encodes all mathematical rules, whether deterministic or stochastic, as probability distributions via energy-based models (EBMs). Deterministic invariants are modeled as sharply peaked distributions, while emergent statistical laws are modeled as broader distributions. This strategy enables direct maximum likelihood learning from observational data, facilitating the seamless modeling of both forms without explicit pre-specification.

Energy-Based Neural Decomposition:

The neural Deflexformer is trained in two phases: (i) blockwise pre-training with synthetic data generated by Deflexpressor to inject mathematical priors, and (ii) full network post-training on observed data. Each block represents a basic mathematical transformation, and their sequential composition enables the modeling of high-order, multiscale relationships. Hierarchical symbolic regression is then performed on intermediate representations extracted from the block outputs, improving interpretability and tractability compared to regression over the end-to-end network.

Lambda Calculus Symbolic Regression:

Moving beyond the limitations of expression trees, the lambda-calculus basis allows Deflexpressor to express variable-arity operations and recursions natively. Type inference ensures only valid, executable code is produced, and heuristics for aggregation initialization bias the search toward plausible scientific hypotheses. With hybrid neural/rule-based expression generation, Deflexpressor efficiently samples high-quality, novel symbolic candidates.

Empirical Evaluation

Deflex was evaluated across a suite of representative simulation and real-world domains: classical particle motion, water particles (with coarse-graining for phase transition), fluid dynamics (2D and 3D turbulence, Navier-Stokes identification), and collective animal and human motion.

Accuracy and Efficiency

Empirical results demonstrate that Deflex is able to recover both canonical equations (e.g., energy/momentum conservation, Langevin, Maxwell-Boltzmann, Navier-Stokes) and distributional laws (e.g., emergence of velocity distributions, power-law, Lévy flights), often with higher accuracy and up to 7x lower computational cost versus state-of-the-art (SOTA) symbolic regression and neural-SR baselines. Notably, for complex equations such as Navier-Stokes in 3D turbulence, Deflex uniquely recovers the complete structure including pressure-field terms—an unattainable result for all tested baselines.

Figure 3: Comparative RMSE/NLL (accuracy) and runtime (efficiency) for Deflex and baselines in discovering diverse laws across domains.

Cross-Scale Robustness

Deflex maintains low earth mover's distance (EMD) between learned and ground-truth distributions across hierarchical coarse-graining scales, demonstrating scale-robust formula recovery where other baselines degrade.

Figure 4: EMD-based cross-scale performance of Deflex versus baseline SR on water and fluid systems; efficiency/accuracy trade-off on Feynman benchmark.

Ablation and Component Analysis

Pre-training volume, block depth, and architectural hyperparameters directly affect formula recovery accuracy and speed; larger synthetic pre-training datasets, increased model depth, and adequate embedding sizes yield improved results, with diminishing returns beyond moderate increases. Langevin sampling for energy-based models is essential for convergence to equilibrium distributions.

Figure 5: Ablation showing improvements in accuracy and recovery rate from increased pre-training, depth, and sampling strategies; validation of conservation laws in learned representations.

Theoretical and Practical Implications

The principal theoretical contribution is a concrete instantiation of unified energy-based modeling for both invariants and distributions, enabling continuous, unsupervised discovery of equations governing multiscale systems. The adoption of lambda calculus for symbolic regression circumvents the combinatorial brittleness of tree-based methods, paving the way for more generalizable and expressive symbolic AI. By integrating neural and symbolic paradigms, Deflex realizes a high degree of autonomy in scientific hypothesis formation, reducing the need for manual priors or structural constraints.

Practically, Deflex enables scalable, interpretable law discovery in domains marked by complexity, heterogeneity, and data scarcity, provided multiscale observation samples can be systematized. The capacity to identify not only standard but also undocumented empirical regularities in systems as varied as molecular dynamics and collective biobehavior demonstrates broad applicability.

Limitations and Future Directions

While Deflex demonstrates strong generality and performance, several limitations remain. There are open challenges in efficiently sampling the vast symbolic space (reducing invalid candidate generation), automatically handling noisy/incomplete data, and balancing representation of physically meaningful low-frequency structures versus dominant approximations in multiscale settings. The integration of automated data processing, adaptive hypothesis generation, and more stringent theoretical analysis of learned energy landscapes (e.g., connections to renormalization group theory) are called out as promising next steps.

Conclusion

Deflex constitutes a substantive advance in automated, interpretable discovery of governing laws in complex multiscale systems by combining unified energy-based neural modeling with lambda-calculus-guided symbolic regression. Its empirical superiority over conventional and state-of-the-art methods in accuracy, scalability, and ability to recover cross-scale structure marks it as a leading solution for scientific AI applications. The underlying methodological framework offers a blueprint for future machine-guided scientific discovery efforts across disciplines characterized by emergent and multiscale phenomena.