Exhaustive Symbolic Regression

• Exhaustive symbolic regression is a deterministic method that fully enumerates all valid expressions in a deliberately bounded search space, ensuring complete coverage of defined models.
• It employs structured search via grammar-based tree enumeration and semantic deduplication to eliminate redundant expressions, thereby enabling objective model ranking using the MDL principle.
• ESR serves as a reproducible baseline across applications such as cosmology and classification, offering interpretable closed-form models and highlighting trade-offs in computational hardness versus completeness.

Searching arXiv for papers on exhaustive symbolic regression and closely related work. arxiv_search(query="^^^^^^^^4\4 Symbolic Regression^^^^^^^^4\4 OR ^^^^^^^^4\4 regression by exhaustive search^^^^^^^^4\4 OR ^^^^^^^^4\4 Symbolic Regression and model selection by minimum description length^^^^^^^^4\4 OR ^^^^^^^^4\4 Inefficiency of Genetic Programming for Symbolic Regression^^^^^^^^4\4 max_results=^^^^4 OR \4\4^^^^) Fetching the most relevant results to ground the article in recent literature. arxiv_search(query="exhaustive symbolic regression", max_results=^^^^4 OR \4\4^^^^) Exhaustive symbolic regression (ESR) is a family of deterministic symbolic-regression methods that replaces stochastic search with systematic enumeration of all valid expressions in a user-defined search space, usually bounded by a fixed tree complexity and a restricted operator basis or context-free grammar. In this setting, symbolic regression is the problem of learning a mathematical expression from data, while ESR seeks to guarantee coverage of the chosen hypothesis class rather than relying on probabilistic discovery. The defining trade-off is explicit in one of the standard formulations: classical genetic programming performs an incomplete search in a rather complete search space, whereas exhaustive symbolic regression performs a complete search in an incomplete search space (&&&^^^^4\4^^^^&&&, &&&^^^^4 OR \4^^^^&&&, &&&^^^^4 OR \4^^^^&&&).

^^^^4 OR \4^^^^. Formal problem and meaning of exhaustiveness

Symbolic regression can be formalized as optimization over a search space of mathematical expressions built from a primitive set. If PRESERVED_PLACEHOLDER_^^^^4\4^^^^ is a set of functions and variables, and PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^ is the set of functions formed by composition of elements of PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^ and their compositions, then for training data

PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^

the symbolic-regression problem is

$$f^\star=\arg\min_{f\in\mathcal{F}} \mathcal{L}\big(\mathbf{y}, f(\mathbf{x})\big).$$

In this formulation, a candidate model is any expression tree or DAG formed by composing the allowed primitives (&&&^^^^4 OR \4^^^^&&&).

Within ESR, “exhaustive” does not denote enumeration of all imaginable formulas. It denotes complete traversal of a finite, deliberately bounded search space: a fixed operator basis, a maximum expression complexity, and a validity criterion for tree structures. In the original ESR formulation, the method systematically and efficiently considers all possible equations made with a given basis set of operators and up to a specified maximum complexity; within that constrained class, and provided parameters are perfectly optimized, it is guaranteed to find the true optimum and a complete function ranking (&&&^^^^4\4^^^^&&&). The broader MDL-based treatment makes the same point in slightly different terms: ESR enumerates all possible expressions built from a user-specified operator basis up to a chosen maximum complexity, fits their parameters, and ranks them objectively (&&&^^^^4 OR \4^^^^&&&).

This bounded notion of exhaustiveness is central. It distinguishes ESR both from heuristic symbolic-regression systems, which may miss good formulas, and from deterministic local-improvement procedures that do not enumerate all expressions of bounded size. A plausible implication is that the term is best reserved for methods with explicit coverage guarantees over a precisely defined search class, rather than for deterministic search in general.

^^^^4 OR \4^^^^. Search-space construction and bounded completeness

The practical content of ESR lies in how the search space is defined before enumeration begins. In one formulation, expressions are represented as rooted trees with nullary, unary, and binary operators. ESR first generates all valid tree shapes at a fixed node count, then decorates those trees with operators from the basis set, and finally removes duplicates and algebraically equivalent variants as far as possible (&&&^^^^4\4^^^^&&&). The same paper emphasizes that complexity is the number of nodes in the tree, so completeness is indexed by a concrete combinatorial bound rather than by an informal notion of simplicity.

Another line of work imposes a context-free grammar specifically to make exhaustive traversal feasible. In that grammar-driven setting, expressions are restricted to an abstract polynomial structure,

$\begin{split} \mathit{Expr} &= c_1 \mathit{Term}_1 + c_2 \mathit{Term}_2 + \ldots + c_n \ \mathit{Term} &= \mathit{Factor}_0 \times \mathit{Factor}_1 \times \ldots \ \mathit{Factor} &\in \{\mathit{variable}, \log(\mathit{variable}), \exp(\mathit{variable}), \sin(\mathit{variable})\}, \end{split}$

with additional syntactical constraints that allow, for example, $xy + xz$ but not $x(y+z)$, allow $e^x e^y$ but not $e^{x+y}$, disallow nested nonlinear functions, and restrict inverse, square root, and cube root to occur at most once per term. The purpose is not to approximate the full symbolic universe, but to define a large yet finite and semantically deduplicated subspace that is exhaustive enough to be useful in practice (&&&^^^^4 OR \4^^^^&&&).

Exhaustive analysis becomes especially informative in limited settings. An equality-saturation-based ESR implementation uses the grammar

PRESERVED_PLACEHOLDER_^^^^4 OR \4\4^^^^

with expression length defined as the number of nodes in the tree, and carries out exhaustive enumeration only for maximum lengths ^^^^4 OR \4\4^^^^ and ^^^^4 OR \4 OR \4^^^^. This paper is explicit that complete exhaustive enumeration is feasible only in limited settings: one-dimensional SR, a small operator set, and short maximum tree length (Kronberger et al., 2024).

These constructions make the same methodological point in different ways. ESR is exhaustive only after the search space has already been compressed by structural decisions. This suggests that the design of the grammar or basis set is not a peripheral implementation detail; it is the condition that makes exhaustiveness meaningful.

^^^^4 OR \4^^^^. Semantic deduplication and canonicalization

A major obstacle to exhaustive symbolic regression is that the raw tree space is highly redundant. Distinct syntax trees can encode the same function through commutativity, associativity, distributivity, algebraic identities, parameter permutations, or reparameterizations. ESR therefore depends not only on enumeration, but on a mechanism for collapsing large equivalence classes of congruent expressions into semantically unique representatives (Kronberger et al., 2024).

In grammar-based exhaustive search, this is handled by semantic hashing. Hashes are computed bottom-up, children of commutative operators are sorted before hashing, associative operators are folded, expressions are simplified using algebraic identities, and previously seen hashes are cached so that semantically equivalent expressions are skipped. The effect is that the search ceases to be a pure tree traversal and becomes a directed graph with merging (&&&^^^^4 OR \4^^^^&&&).

Equality saturation pushes the same idea further. The improved ESR implementation described in the analysis of GP inefficiency generates candidate trees, rewrites and simplifies them using equality saturation, extracts a canonical representative, hashes that representative, and discards future candidates with the same hash. The paper distinguishes sharply between structurally different expressions and semantically equivalent or congruent expressions, and defines the ESR search space as the set of canonical representatives of equivalence classes rather than the full syntactic tree space (Kronberger et al., 2024).

The reduction can be substantial. For the grammar used in that study, there are about 8^^^^4\4^^^^,4^^^^4\4 unique expressions after simplification at maximum length ^^^^4 OR \4\4^^^^ and about ^^^^4 OR \4^^^^,^^^^4\4^^^^8 OR \4^^^^,8^^^^4\4 OR \4^^^^ at maximum length ^^^^4 OR \4 OR \4^^^^; at length ^^^^4 OR \4\4^^^^ there are about PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^ more trees than unique expressions. The authors also note that equality saturation is not guaranteed to fully saturate all equivalences because they stop after a limited number of iterations, so deduplication is strong but not perfect (Kronberger et al., 2024). Similar caution appears in the original ESR paper, which states that duplicate-removal machinery helps considerably but does not find every duplicate (&&&^^^^4\4^^^^&&&).

The conceptual consequence is straightforward: exhaustive SR is rarely exhaustive over syntax. It is more usefully understood as exhaustive over a canonicalized semantic search space, with completeness defined relative to the available rewrite system.

4. Parameter fitting and model selection by minimum description length

Exhaustive symbolic regression separates structural search from continuous parameter estimation. Once a symbolic structure has been generated, coefficients or other free constants must be fitted numerically. In the grammar-driven deterministic method, every complete sentence undergoes nonlinear least-squares fitting with the Levenberg–Marquardt algorithm, repeated ten times with different initializations to reduce local-minimum failures (&&&^^^^4 OR \4^^^^&&&). In the original ESR implementation, the guarantee of optimality is explicitly conditional on exact parameter optimization; in practice, the paper uses BFGS with multiple restarts and stopping heuristics (&&&^^^^4\4^^^^&&&). A recent application to SDSS DR^^^^4 OR \4^^^^7 uses L-BFGS with ^^^^4 OR \4\4^^^^ multi-start initializations per expression and minimizes

PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^

for each candidate expression (&&&^^^^4 OR \4^^^^6&&&).

What distinguishes ESR from many earlier exact-search proposals is not only exhaustive generation but also rigorous model selection. The central MDL decomposition is

PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^

where PRESERVED_PLACEHOLDER_^^^^4 OR \44^^^^ is the codelength of the function and PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^5 is the codelength of the residuals. Under Shannon–Fano coding,

PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^6

The final MDL score used in the ESR framework is

PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^7

where PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^8 is the number of tree nodes, PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^9 is the number of operators, PRESERVED_PLACEHOLDER_^^^^4 OR \4\4^^^^ is the number of free parameters, PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^ are integer constants arising in simplification, and PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^ is the observed Fisher information matrix (&&&^^^^4\4^^^^&&&, &&&^^^^4 OR \4^^^^&&&).

The Fisher-information term determines the optimal precision for encoding each parameter,

PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^

so the data determine how precisely a parameter must be transmitted. This makes MDL a principled way to place goodness of fit and structural complexity on the same information-theoretic scale (&&&^^^^4 OR \4^^^^&&&). The same paper also introduces a Katz back-off language-model prior as an alternative complexity prior learned from a training set of successful equations in a domain.

MDL is therefore not an auxiliary scoring heuristic. In ESR, it is the mechanism that turns exhaustive search into a ranked scientific inference procedure.

5. Computational hardness, scaling, and common misconceptions

Symbolic regression is NP-hard. The formal hardness result proceeds through the decision version of SR and a polynomial-time reduction from the decision version of the unbounded subset sum problem, yielding Theorem ^^^^4 OR \4^^^^: the SR problem is NP-hard (&&&^^^^4 OR \4^^^^&&&). The paper’s practical interpretation is explicit: exact search over all symbolic expressions is worst-case intractable, and a polynomial-time exact SR algorithm for the general problem would imply PRESERVED_PLACEHOLDER_^^^^4 OR \44^^^^. Exhaustive or complete methods must therefore rely on heavy restrictions, pruning, or bounded model classes to remain tractable.

The scaling behavior of ESR is consistent with that result. In the MDL-focused ESR treatment, runtime scales exponentially with complexity, a typical full run reaches about complexity ^^^^4 OR \4\4^^^^ and takes roughly ^^^^4 OR \4\4\4^^^^ CPU-hours, more operators mean more expressions and thus a lower reachable complexity, and the process is only partly parallelizable because parameter fitting is embarrassingly parallel whereas simplification and deduplication require comparisons between functions (&&&^^^^4 OR \4^^^^&&&). The original ESR cosmology study provides a concrete illustration: up to complexity ^^^^4 OR \4\4^^^^, the search involved about 5.^^^^4 OR \4^^^^ million trial functions, reduced to ^^^^4 OR \4 OR \44^^^^,^^^^4 OR \4 OR \44^^^^ unique equations, of which ^^^^4 OR \4 OR \4^^^^9,86^^^^4 OR \4^^^^ contained parameters and required optimization (&&&^^^^4\4^^^^&&&).

Several misconceptions follow from overlooking these bounds. First, deterministic symbolic regression is not automatically exhaustive. A deterministic tree-growing method based on semantic-space error reduction evaluates candidate substitutions analytically and is reproducible, but it does not enumerate all expressions within a size bound and does not guarantee the globally best expression in that bounded space (&&&^^^^4 OR \4 OR \4^^^^&&&). Second, hybrid enumeration methods are often exhaustive only over a pruned and bounded class. The mixed-integer nonlinear optimization approach that enumerates generalized expression trees and solves each with BARON is exhaustive over the enumerated, pruned, bounded search space, but not globally complete over all symbolic expressions (&&&^^^^4 OR \44^^^^&&&).

A plausible implication is that the phrase “exhaustive symbolic regression” should be interpreted operationally, not metaphysically. It refers to exact coverage of a finite search space whose boundaries are themselves part of the method.

6. Scientific applications

The original ESR application to cosmology studied the expansion history of the Universe by modeling PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^5 from ^^^^4 OR \4 OR \4^^^^ cosmic chronometer measurements and the Pantheon+ supernova sample. Using the basis set PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^6 and maximum complexity ^^^^4 OR \4\4^^^^, ESR found about 4^^^^4\4^^^^ functions out of 5.^^^^4 OR \4^^^^ million trial functions that fit the data more economically than the Friedmann equation. For cosmic chronometers, the best MDL-ranked function was PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^7; for Pantheon+, the best was PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^8. The paper’s conclusion was not that PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^9CDM is false, but that low-redshift data do not uniquely prefer the expansion history of the standard model of cosmology under MDL (&&&^^^^4\4^^^^&&&).

A later MDL-centered treatment extends this style of analysis to three astrophysical problems: the expansion history of the universe, the effective behaviour of gravity in galaxies, and the potential of the inflaton field. For PRESERVED_PLACEHOLDER_^^^^4 OR \4\4^^^^, the MDL-preferred functions have complexity 5 and are simpler than the Friedmann equations at complexity 7; the paper reports ^^^^4 OR \4^^^^8 functions better than the Friedmann equation for cosmic chronometers and ^^^^4 OR \4^^^^6 better for supernovae. For the radial acceleration relation, ESR finds many functions better than MOND interpolating functions, although the paper stresses that the present data are not sufficient to discriminate strongly on the low-acceleration asymptotic behavior. For inflaton potentials, the MDL-best function under the default structural prior is PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^, whereas replacing the prior with the Katz back-off prior yields more physically reasonable forms such as PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^ and PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^ (&&&^^^^4 OR \4^^^^&&&).

ESR has also been used as a deterministic symbolic-regression baseline in classification. In the SDSS DR^^^^4 OR \4^^^^7 star/galaxy/quasar study, ESR searches over expressions built from binary operators PRESERVED_PLACEHOLDER_^^^^4 OR \44^^^^, unary operator PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^5, and terminals PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^6 and scalar constants, with filters requiring dependence on PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^7, disallowing nested exponentials, allowing only physically plausible exponential motifs such as PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^8, and capping complexity at PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^9. The discovered ESR score function was

$$f^\star=\arg\min_{f\in\mathcal{F}} \mathcal{L}\big(\mathbf{y}, f(\mathbf{x})\big).$$^^^^4\4^^^^

Evaluated by 5-fold stratified cross-validation on ^^^^4 OR \4\4\4^^^^,^^^^4\4\4\4^^^^ spectroscopically confirmed objects, ESR achieved Cohen’s $$f^\star=\arg\min_{f\in\mathcal{F}} \mathcal{L}\big(\mathbf{y}, f(\mathbf{x})\big).$$^^^^4 OR \4^^^^, accuracy $$f^\star=\arg\min_{f\in\mathcal{F}} \mathcal{L}\big(\mathbf{y}, f(\mathbf{x})\big).$$^^^^4 OR \4^^^^, balanced accuracy $$f^\star=\arg\min_{f\in\mathcal{F}} \mathcal{L}\big(\mathbf{y}, f(\mathbf{x})\big).$$^^^^4 OR \4^^^^, and macro-F^^^^4 OR \4^^^^ $$f^\star=\arg\min_{f\in\mathcal{F}} \mathcal{L}\big(\mathbf{y}, f(\mathbf{x})\big).$$4 (&&&^^^^4 OR \4^^^^6&&&).

These applications show two recurring features of ESR. It produces closed-form models that are compact enough to be inspected directly, and its conclusions are often framed cautiously: a model may be MDL-preferred or empirically economical without being physically privileged.

7. Comparative evaluations and position within the symbolic-regression ecosystem

Exhaustive symbolic regression is frequently used as a benchmark for understanding heuristic search rather than merely as an end-user solver. The equality-saturation study comparing GP with exhaustive baselines constructs the full space of semantically unique expressions in limited settings and then measures how GP behaves inside that space. On the Nikuradse dataset, only about ^^^^4 OR \4\4^^^^–^^^^4 OR \4\4^^^^% of all evaluated expressions are unique over whole GP runs, and about 5^^^^4\4^^^^% of expressions at length ^^^^4 OR \4\4^^^^ and ^^^^4 OR \4 OR \4^^^^ simplify to the trivial constant model $$f^\star=\arg\min_{f\in\mathcal{F}} \mathcal{L}\big(\mathbf{y}, f(\mathbf{x})\big).$$5; on the radial acceleration dataset, only about ^^^^4 OR \4\4^^^^% of evaluated expressions are unique, so about 9^^^^4\4^^^^% are revisits. In both tasks, idealized random search over unique expressions outperforms GP when efficiency is measured by visited expressions, and GP does not find the global optima identified by ESR in any of the reported 5^^^^4\4^^^^ runs (Kronberger et al., 2024).

In direct comparisons with modern symbolic-regression systems, ESR usually occupies the role of a reproducible, classically interpretable baseline. In SDSS DR^^^^4 OR \4^^^^7 star/galaxy/quasar separation, ESR is better than PySR on all reported classification metrics, worse than PhySO and MvSR, and still above the $$f^\star=\arg\min_{f\in\mathcal{F}} \mathcal{L}\big(\mathbf{y}, f(\mathbf{x})\big).$$6 range for a compact one-dimensional symbolic model. The paper characterizes ESR as the strongest “classically interpretable” deterministic baseline, but not the top performer overall (&&&^^^^4 OR \4^^^^6&&&).

ESR has also been examined as a source of initial individuals for GP. In a study using Operon’s NSGA-II implementation, ESR-based initialization is built by running ESR with maximum complexities $$f^\star=\arg\min_{f\in\mathcal{F}} \mathcal{L}\big(\mathbf{y}, f(\mathbf{x})\big).$$7, taking the ^^^^4 OR \4exhaustive symbolic regression4\4^^^^ most accurate models from each run, and retaining ESR-fitted parameters in the initial GP population. The main result is that ESR initialization gives an initial advantage that disappears after only a few generations; the authors find no significant differences in accuracy or model complexity among initialization methods and conclude that, given similar diversity in the initial population, the effect of the initialization method on the final Pareto front is negligible (&&&^^^^4 OR \4\4^^^^&&&).

The broader symbolic-regression literature positions ESR against several alternatives. Uniform random global search is proposed as a control algorithm and is described as reducing to an exhaustive search only in the asymptotic sense over a finite bounded space (&&&^^^^4 OR \4 OR \4^^^^&&&). Guided reinforcement-learning systems such as EGRL-SR are presented explicitly as alternatives to exhaustive search in practice rather than as literal brute-force enumeration (&&&^^^^4 OR \4 OR \4^^^^&&&). Likewise, RSRM is described not as strict exhaustive search but as a guided, near-exhaustive search framework over symbolic expression spaces, constrained by learned Q-values, MCTS heuristics, and discovered sub-tree forms (&&&^^^^4 OR \4 OR \4^^^^&&&).

Taken together, these comparisons define ESR’s distinctive role. It is not the universal solution to symbolic regression, nor is it scalable in the unrestricted sense. Its value lies in providing complete coverage, semantic transparency, and principled ranking within a bounded search class, and in supplying a reference standard against which stochastic, random, and learned search procedures can be evaluated.