ParFam Architecture: Symbolic Regression

• ParFam architecture is a symbolic regression model that uses a single-hidden-layer network with trainable rational functions and fixed unary base activations.
• It replaces combinatorial expression-tree search with continuous parameter fitting, achieving provable C¹ approximation guarantees and efficient PDE recovery.
• The design employs sparsity-inducing regularization and skip connections to enhance interpretability, convergence, and physical-law learning.

Searching arXiv for ParFam architecture and related approximation/symbolic regression papers. ParFam, short for “Parametric-Family,” is a symbolic-regression architecture that represents formulas by a single-hidden-layer feed-forward network whose trainable maps are rational functions and whose hidden units apply fixed unary base functions. In its canonical form, the architecture combines a rational encoder, coordinate-wise activations, a rational decoder, and a rational skip connection from input to output. This construction was introduced as a way to replace discrete expression-tree search by continuous parameter fitting in symbolic regression, and subsequent approximation-theoretic work showed that ParFam-type networks admit provable $\mathcal{C}^1$ approximation guarantees for suitably regular target functions (Scholl et al., 2023, Morina et al., 27 Aug 2025).

1. Formal definition and architectural structure

In the formulation summarized by Scholl et al., ParFam is a single-hidden-layer feed-forward network in which the maps into and out of the hidden layer are general multidimensional rational functions, while the hidden layer applies arbitrary unary “base” activations coordinate-wise. If $r_1:\mathbb{R}^d\to\mathbb{R}^h$ denotes the rational encoder into $h$ hidden units, $\sigma:\mathbb{R}^h\to\mathbb{R}^h$ denotes the component-wise activations $\sigma_j$, and $r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$ denotes the rational decoder acting on the concatenated vector $(\sigma(r_1(x)),x)$, then the ParFam map is

$$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$

Each component function $r_\ell$ is rational,

$$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$

The positivity condition on the denominator excludes poles on the domain of interest and is structurally important in both analysis and optimization (Morina et al., 27 Aug 2025).

ParFam is also a special case of the more general symbolic-regression architecture

$r_1:\mathbb{R}^d\to\mathbb{R}^h$0

with $r_1:\mathbb{R}^d\to\mathbb{R}^h$1, $r_1:\mathbb{R}^d\to\mathbb{R}^h$2, $r_1:\mathbb{R}^d\to\mathbb{R}^h$3 the encoder, and $r_1:\mathbb{R}^d\to\mathbb{R}^h$4 the decoder including the skip connection. In the symbolic-regression interpretation, one may interpret $r_1:\mathbb{R}^d\to\mathbb{R}^h$5 as learning a rational embedding of the inputs into a latent space of dimension $r_1:\mathbb{R}^d\to\mathbb{R}^h$6, $r_1:\mathbb{R}^d\to\mathbb{R}^h$7 as applying simple univariate “dictionary” functions coordinate-wise, and $r_1:\mathbb{R}^d\to\mathbb{R}^h$8 as fitting the final combination in rational form. The skip connection exposes the original input directly to the final rational map and makes the architecture resemble a one-hidden-layer residual network with rational, rather than affine, trainable layers.

The activation dictionary is not fixed by the architecture. The summaries explicitly mention examples such as $r_1:\mathbb{R}^d\to\mathbb{R}^h$9, $h$0, and $h$1; in one PDE-recovery implementation, $h$2 consists of element-wise sine and exponential. This design separates the symbolic primitives from the trainable rational maps.

2. ParFam as a continuous symbolic-regression model

The central methodological idea of ParFam is to replace combinatorial search over expression trees by fitting an explicit parametric family of symbolic functions with continuous parameters. In the notation of the symbolic-regression paper, one chooses $h$3 unary base functions $h$4 and trainable rational functions $h$5, and defines

$h$6

This is the symbolic-regression form of the same one-hidden-layer architecture: the $h$7 play the role of rational layers, the $h$8 are the hidden-unit base functions, and $h$9 receives both transformed hidden features and the original input through the skip pathway (Scholl et al., 2023).

Each trainable rational map is parameterized by numerator and denominator polynomials of fixed maximum degree. For a component $\sigma:\mathbb{R}^h\to\mathbb{R}^h$0 of a rational transform $\sigma:\mathbb{R}^h\to\mathbb{R}^h$1,

$\sigma:\mathbb{R}^h\to\mathbb{R}^h$2

and

$\sigma:\mathbb{R}^h\to\mathbb{R}^h$3

In the PDE framework, the denominator-coefficient vector is normalized by $\sigma:\mathbb{R}^h\to\mathbb{R}^h$4 and constrained to a closed positive subset $\sigma:\mathbb{R}^h\to\mathbb{R}^h$5 ensuring $\sigma:\mathbb{R}^h\to\mathbb{R}^h$6.

Given data $\sigma:\mathbb{R}^h\to\mathbb{R}^h$7, ParFam fits parameters by minimizing a regularized mean-squared error,

$\sigma:\mathbb{R}^h\to\mathbb{R}^h$8

The $\sigma:\mathbb{R}^h\to\mathbb{R}^h$9 penalty encourages sparsity in the polynomial coefficients and thereby suppresses unused monomial terms. This is the mechanism by which the continuous fit is pushed toward a compact symbolic form.

Because the loss is highly nonconvex, the original ParFam method uses basin-hopping combined with local search. The described procedure alternates randomized jumps in parameter space with BFGS-based local minimization, followed by a “drop-to-zero” sparsification step in which small coefficients are thresholded to zero and the remaining parameters are refined by L-BFGS. A further extension, DL-ParFam, employs a pre-trained Set Transformer to predict promising model-parameter choices—degrees and base-function selections—from data, reducing the number of basin-hopping runs from $\sigma_j$0 to $\sigma_j$1 and yielding speedups of $\sigma_j$2–$\sigma_j$3. The symbolic-regression paper reports extensive numerical experiments on SRBench and states that the method achieves state-of-the-art results.

3. $\sigma_j$4-approximation theory

The approximation-theoretic analysis places ParFam within a broader class of rational neural networks and shows that the architecture is not merely expressive in an empirical sense but admits quantitative $\sigma_j$5 approximation rates. Let $\sigma_j$6, $\sigma_j$7, and suppose

$\sigma_j$8

Then for any $\sigma_j$9 there exists a family of ParFam networks $r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$0 of width $r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$1, depth $r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$2 (one hidden layer plus output layer), and maximum rational degree $r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$3 such that

$r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$4

for a constant $r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$5. In particular, as $r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$6, the networks approximate both $r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$7 and $r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$8 uniformly (Morina et al., 27 Aug 2025).

A corollary yields the same $r_2:\mathbb{R}^{h+d}\to\mathbb{R}^p$9 rate for a single rational function, interpreted as a depth-$(\sigma(r_1(x)),x)$0 ParFam network with zero hidden units. More precisely, one obtains a rational function $(\sigma(r_1(x)),x)$1 of total degree $(\sigma(r_1(x)),x)$2 satisfying

$(\sigma(r_1(x)),x)$3

Thus the hidden layer is not required for approximation in principle, but it provides a structured factorization of the rational approximation.

The proof strategy is a transfer argument from ReQU approximation theory. The starting point is a higher-order universal approximation theorem of Belomestny et al. for ReQU networks, which gives $(\sigma(r_1(x)),x)$4 approximation of $(\sigma(r_1(x)),x)$5 at rate $(\sigma(r_1(x)),x)$6 by a constant-depth network with Rectified Quadratic Unit activations and width $(\sigma(r_1(x)),x)$7. Each ReQU activation on $(\sigma(r_1(x)),x)$8 is then approximated in $(\sigma(r_1(x)),x)$9 by a univariate rational function $$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$0 using Newman’s rational approximants; the error in both function and derivative can be made $$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$1 for arbitrary $$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$2. Replacing every ReQU in the spline-based realization by such rational approximants and controlling degrees, widths, and Lipschitz constants yields a rational network of the same shape with the stated rate, provided $$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$3.

The one-dimensional illustrative case makes the rate transparent. For $$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$4, $$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$5, and a ParFam network with $$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$6 hidden units and base activations $$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$7 and $$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$8, the general theorem implies that, for suitable numerator and denominator degrees growing like $$\Phi(x)=r_2\bigl(\sigma(r_1(x)),x\bigr)\in\mathbb{R}^p .$$9, any sufficiently smooth target can be fitted in $r_\ell$0 at rate $r_\ell$1.

4. Regularization, sparsity, and identifiability

ParFam’s symbolic character depends not only on expressive power but also on how parameters are selected among many equivalent or near-equivalent rational representations. The PDE-recovery framework makes this explicit through an all-at-once inverse-problem formulation with regularization

$r_\ell$2

The $r_\ell$3 term drives many numerator and denominator coefficients to zero and thereby prunes unused monomials. The denominator coefficients are simultaneously constrained to the unit-norm positive set $r_\ell$4 so that no poles appear and the scaling invariance between numerator and denominator is removed (Morina et al., 17 Feb 2026).

The identifiability theorem in this framework concerns recovery of an unknown PDE law $r_\ell$5 represented by a symbolic network $r_\ell$6. Under the assumptions that the true state $r_\ell$7 lies in the Banach space $r_\ell$8 and satisfies $r_\ell$9, that $$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$0 is exactly representable by the architecture, that the measurement operators $$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$1 converge to the full operator $$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$2, and that the regularization parameters satisfy $$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$3, $$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$4, and $$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$5, any minimizers $$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$6 of the all-at-once problem admit a subsequence $$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$7 such that

$$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$8

and $$r_\ell(z)=\frac{p_\ell(z)}{q_\ell(z)}, \qquad p_\ell,q_\ell \text{ polynomials with } q_\ell>0, \qquad \deg(p_\ell),\deg(q_\ell)\le N_\ell .$$9 converges weakly to $r_1:\mathbb{R}^d\to\mathbb{R}^h$00 in $r_1:\mathbb{R}^d\to\mathbb{R}^h$01. Moreover, $r_1:\mathbb{R}^d\to\mathbb{R}^h$02 minimizes the regularizer among all admissible representations.

The corresponding corollary isolates the condition under which symbolic recovery becomes unique. If, in addition,

$r_1:\mathbb{R}^d\to\mathbb{R}^h$03

so that no two distinct networks agree on the data manifold, then the full sequence satisfies $r_1:\mathbb{R}^d\to\mathbb{R}^h$04 and one recovers the exact coefficients of the simplest PDE law. A common misconception is that sparsity alone guarantees uniqueness. The identifiability result shows that this is not the case: sparsity selects a preferred representation, but exact coefficient recovery requires the extra structural identifiability condition.

5. Use in PDE recovery and physical-law learning

The PDE-recovery study provides a concrete ParFam implementation within an inverse-problem pipeline. The measurement grid consists of $r_1:\mathbb{R}^d\to\mathbb{R}^h$05 spatial points and $r_1:\mathbb{R}^d\to\mathbb{R}^h$06 time points on $r_1:\mathbb{R}^d\to\mathbb{R}^h$07. Measurements are frequency-truncated: $r_1:\mathbb{R}^d\to\mathbb{R}^h$08 retains the lowest $r_1:\mathbb{R}^d\to\mathbb{R}^h$09 Fourier modes in time-blocks of width $r_1:\mathbb{R}^d\to\mathbb{R}^h$10. The noise model is multiplicative,

$r_1:\mathbb{R}^d\to\mathbb{R}^h$11

For fixed $r_1:\mathbb{R}^d\to\mathbb{R}^h$12, the regularizer weights are

$r_1:\mathbb{R}^d\to\mathbb{R}^h$13

In the reported ParFam configuration, the architecture has one hidden layer, with widths $r_1:\mathbb{R}^d\to\mathbb{R}^h$14 for the inputs $r_1:\mathbb{R}^d\to\mathbb{R}^h$15 and $r_1:\mathbb{R}^d\to\mathbb{R}^h$16 in practice; the numerator-polynomial degree is $r_1:\mathbb{R}^d\to\mathbb{R}^h$17, the denominator degree is $r_1:\mathbb{R}^d\to\mathbb{R}^h$18, and the activation set is a mix of element-wise sine and exponential (Morina et al., 17 Feb 2026).

Optimization alternates between state updates and law updates. The $r_1:\mathbb{R}^d\to\mathbb{R}^h$19-update uses basin-hopping plus BFGS for $r_1:\mathbb{R}^d\to\mathbb{R}^h$20 iterations, while the $r_1:\mathbb{R}^d\to\mathbb{R}^h$21-update uses ADAM with learning rate $r_1:\mathbb{R}^d\to\mathbb{R}^h$22, up to $r_1:\mathbb{R}^d\to\mathbb{R}^h$23 epochs, and early stopping. The alternating scheme repeats the two substeps for $r_1:\mathbb{R}^d\to\mathbb{R}^h$24 full cycles. This combines the global nonconvex search strategy inherited from ParFam with gradient-based optimization of the latent state.

Two one-dimensional PDE examples are reported. In the uniquely identifiable case,

$r_1:\mathbb{R}^d\to\mathbb{R}^h$25

The monitored errors are $r_1:\mathbb{R}^d\to\mathbb{R}^h$26 and $r_1:\mathbb{R}^d\to\mathbb{R}^h$27, and both decrease monotonically as $r_1:\mathbb{R}^d\to\mathbb{R}^h$28 increases from $r_1:\mathbb{R}^d\to\mathbb{R}^h$29 to $r_1:\mathbb{R}^d\to\mathbb{R}^h$30. At $r_1:\mathbb{R}^d\to\mathbb{R}^h$31, the learned law is

$r_1:\mathbb{R}^d\to\mathbb{R}^h$32

which is reported as a near-exact sparse reconstruction of $r_1:\mathbb{R}^d\to\mathbb{R}^h$33.

In the non-identifiable case, the law family is

$r_1:\mathbb{R}^d\to\mathbb{R}^h$34

The same error metrics again decrease. The learned law is reported as an $r_1:\mathbb{R}^d\to\mathbb{R}^h$35-minimal representative from the non-unique solution set, for example

$r_1:\mathbb{R}^d\to\mathbb{R}^h$36

These experiments support the theoretical distinction between convergence of the recovered law on the data manifold and uniqueness of the symbolic form.

6. Interpretability, smoothness, and architectural scope

ParFam is explicitly designed for settings in which the learned model should remain symbolically inspectable after training. Every nonlinear step is represented in rational form, both in the map into the latent symbolic basis and in the final combination. If many denominator coefficients become negligible or joint factors emerge, the learned expression can be simplified symbolically. This is one reason the architecture is emphasized in symbolic regression for physical-law learning (Morina et al., 27 Aug 2025).

The same summary attributes several computational and analytical benefits to the rational formulation. Because rational functions and their derivatives are analytic on compact domains when the denominator is positive, ParFam networks admit stable gradient flows and deliver built-in $r_1:\mathbb{R}^d\to\mathbb{R}^h$37 smoothness away from poles. The skip connection exposing the input $r_1:\mathbb{R}^d\to\mathbb{R}^h$38 directly to the final rational map is stated to accelerate convergence in regression tasks, since it allows the network to model additive “known” physics plus a small corrective rational term. From the approximation results, ParFam-type networks inherit the same provable higher-order convergence rates in $r_1:\mathbb{R}^d\to\mathbb{R}^h$39 that ReQU spline networks enjoy, but in a rational-network form that is more amenable to symbolic extraction of compact formulas.

At the same time, the architecture has a sharply defined scope. In the papers summarized here, ParFam itself is a one-hidden-layer model, even though it sits inside a more general compositional class of symbolic networks. Its interpretability is therefore linked to constrained depth, explicit rational parameterization, fixed unary dictionaries, and sparsity-inducing regularization rather than to unrestricted neural-network scaling. A plausible implication is that ParFam occupies a specific point in the design space between classical symbolic regression and general-purpose deep learning: it retains continuous optimization and trainable latent representations, but its structural bias is toward compact rational formulas with explicit unary primitives.