Symbolic Quantile Regression

• Symbolic Quantile Regression (SQR) is an extension of symbolic regression that estimates conditional quantiles with the pinball loss, providing clear, interpretable models for various parts of a response distribution.
• It employs a multi-objective optimization that balances predictive accuracy and token complexity by constructing a Pareto front of candidate symbolic expressions.
• Empirical results show that SQR outperforms traditional transparent models in normalized quantile loss and parsimony, offering actionable insights in applications like fuel consumption analysis.

Searching arXiv for the specified paper and closely related quantile-regression uses of the acronym SQR. Symbolic Quantile Regression (SQR) is an extension of Symbolic Regression (SR) from conditional-mean prediction to conditional-quantile prediction. Rather than estimating only $E[Y\mid X]$, it targets the conditional quantile function

$$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$

thereby enabling white-box prediction of the median, upper-tail, or other distributional functionals of the target variable. In the formulation introduced in "Symbolic Quantile Regression for the Interpretable Prediction of Conditional Quantiles" (Hoekstra et al., 11 Aug 2025), candidate predictors are symbolic expressions optimized under the pinball loss while simultaneously controlled for expression complexity, so that predictive performance and parsimony are treated as joint objectives rather than collapsed into a single scalar criterion.

1. Formal definition and objective

The defining shift from standard symbolic regression to SQR is the replacement of conditional-mean estimation with quantile estimation. The target is the $\tau$-quantile of $Y$ conditional on $X=x$, with $\tau\in(0,1)$. This directly addresses settings in which the center of the distribution is insufficient, especially when interest lies in the median or in extreme outcomes.

The loss function is the pinball loss,

$$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$

or equivalently

$$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$

Within SQR, the predictive model $f$ is a symbolic expression assembled from a library of tokens including features $x_j$, real constants, and operators such as $$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$0 (Hoekstra et al., 11 Aug 2025). Internally, these expressions are represented as syntax trees or expression graphs.

The paper frames SQR as a trade-off between predictive performance and interpretability. Interpretability is operationalized through token-complexity: if $$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$1 denotes the cost of token $$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$2, then the complexity of an expression is

$$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$3

For each allowed complexity level $$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$4, SQR solves

$$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$5

and then collects the Pareto front

$$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$6

This formulation makes SQR a genuinely multi-objective approach. A plausible implication is that model selection can remain aligned with domain-specific transparency constraints rather than being fixed by a globally chosen regularization coefficient.

2. Symbolic representation, search, and optimization

In the reported implementation, the base engine is PySR, combining multi-population genetic programming with constant optimization (Hoekstra et al., 11 Aug 2025). Initialization uses random expression trees up to a maximum initial size or depth. Variation proceeds through crossover and mutation: crossover swaps subtrees between parent expressions, while mutation may replace an operator, insert or delete a node, or perturb a constant.

Fitness evaluation uses the empirical pinball loss as the primary objective. A secondary mechanism, described as an adaptive parsimony penalty, encourages exploration of underrepresented complexities. This is paired with local optimization: once a symbolic structure has been selected, BFGS refines its real-valued constants to minimize pinball loss.

The search process also incorporates diversity-preserving mechanisms. The paper lists age-based regularization, under which younger individuals are temporarily favored, and simulated annealing, which accepts poorer individuals early in the search to avoid premature convergence. Constraints include a maximum tree size, exemplified as $$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$7 nodes, a maximum depth, and optional user constraints on variables or admissible functional forms.

These design choices position SQR within the broader SR tradition of evolutionary search plus local refinement, but with the quantile loss replacing the more familiar squared-error or mean-oriented objectives. This suggests that the central algorithmic novelty is not a new symbolic search formalism, but the systematic adaptation of the SR pipeline to quantile-specific risk minimization.

3. Parsimony, Pareto optimality, and model selection

A central feature of SQR is the explicit separation of predictive quality from expression simplicity. Rather than blending the two into a single penalized objective, SQR constructs a Pareto front over pinball loss and token-complexity (Hoekstra et al., 11 Aug 2025). The result is a family of nondominated models spanning simpler and more complex expressions.

The complexity metric is defined heuristically through token costs, with examples such as $$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$8 costing $$Q_Y(\tau\mid X=x)\;:=\;\inf\{q\in\mathbb R:\Pr(Y\le q\mid X=x)\ge\tau\},\quad\tau\in(0,1),$$9, $\tau$0 costing $\tau$1, and $\tau$2 costing $\tau$3. The paper identifies interpretability with parsimony measured by $\tau$4, while also noting that the complexity metric is only a heuristic proxy for interpretability. This is an important qualification: low token-complexity does not guarantee semantic ease of interpretation, but it provides an operational criterion suitable for optimization.

Final model selection is performed from the Pareto front according to user needs. The paper explicitly mentions the “elbow” rule, a maximum allowed complexity, or domain-specific interpretability requirements. In practice, this means that SQR does not yield a single canonical formula per quantile; it yields a structured set of candidate formulas whose suitability depends on the application’s tolerance for algebraic complexity.

A common misconception is that interpretability in SR is equivalent to linearity or monotonicity. The SQR formulation does not require either property. Instead, interpretability is tied to a constrained symbolic vocabulary and low expression complexity, allowing nonlinear and interaction-rich quantile models while preserving white-box access to the final analytic form.

4. Empirical evaluation and benchmark results

The empirical evaluation uses $\tau$5 noisy regression problems drawn from SRBench/PMLB, including real and synthetic datasets with dimensions ranging from a few to hundreds of features (Hoekstra et al., 11 Aug 2025). The reported baselines are Linear Quantile Regressor (LQR), Quantile Decision Tree (QDT), LightGBM Quantile Regressor (LGBM), SQR, and SQR10K, where SQR10K denotes a version subsampled to $\tau$6 points.

Evaluation is averaged over $\tau$7-fold cross-validation and then over datasets. The metrics are normalized quantile loss (nql), defined as pinball loss divided by $\tau$8; absolute coverage error (ace), defined as $\tau$9; parsimony, measured by average $Y$0; and runtime.

Metric	LQR / QDT / LGBM	SQR / SQR10K
nql ($Y$1)	LQR $Y$2, QDT $Y$3, LGBM $Y$4	SQR $Y$5, SQR10K $Y$6
nql ($Y$7)	LQR $Y$8, QDT $Y$9, LGBM $X=x$0	SQR $X=x$1, SQR10K $X=x$2
ace ($X=x$3)	LQR $X=x$4, QDT $X=x$5, LGBM $X=x$6	SQR $X=x$7, SQR10K $X=x$8
ace ($X=x$9)	LQR $\tau\in(0,1)$0, QDT $\tau\in(0,1)$1, LGBM $\tau\in(0,1)$2	SQR $\tau\in(0,1)$3, SQR10K $\tau\in(0,1)$4
parsimony ($\tau\in(0,1)$5)	LQR $\tau\in(0,1)$6, QDT $\tau\in(0,1)$7	SQR $\tau\in(0,1)$8, SQR10K $\tau\in(0,1)$9
parsimony ($$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$0)	LQR $$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$1, QDT $$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$2	SQR $$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$3, SQR10K $$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$4

The paper states that statistical tests confirm SQR significantly outperforms other transparent models on nql and parsimony, and on ace at $$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$5 (Hoekstra et al., 11 Aug 2025). Runtime is reported as approximately $$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$6–$$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$7 seconds per dataset for SQR and approximately $$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$8 seconds for SQR10K, while the baselines run in milliseconds.

These results matter for two reasons. First, they show that white-box quantile models need not incur a large accuracy penalty relative to stronger baselines. Second, they indicate that the parsimony objective can produce formulas substantially smaller than those of the transparent comparators used in the study, especially QDT. A plausible implication is that the main empirical niche of SQR is not merely interpretability in isolation, but interpretability under competitive quantile performance.

5. Airline fuel-usage case study

The paper’s case study uses Boeing 777 flights over 2019–2023 and four features: Adjusted Speeding Factor (ASF), Great-circle distance (GCD), Average wind component (AWC), and Total passengers (TP) (Hoekstra et al., 11 Aug 2025). ASF is defined as a departure delay-driven speedup variable, and AWC is signed so that headwind $$L_{\tau}(y,f(x)) \;=\; \begin{cases} \tau\,(y-f(x)), & y\ge f(x),\ (\tau-1)\,(y-f(x)), & y< f(x), \end{cases}$$9 and tailwind $$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$0.

For the median, the reported model is

$$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$1

Its reported coverage is approximately $$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$2, with pinball loss approximately $$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$3. The interpretation given in the paper is that fuel is approximately linear in distance, with weak interaction terms involving passenger load and wind.

For the upper quantile, the reported $$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$4 model is

$$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$5

Its reported coverage is approximately $$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$6, with pinball loss approximately $$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$7. The accompanying interpretation is that extreme fuel use is dominated by speeding behavior, with distance, load, and wind entering through a rational interaction.

The case study is significant because it illustrates how different quantiles can induce qualitatively different symbolic structures. The median model emphasizes distance with modest interactions, whereas the upper-quantile model elevates ASF to a dominant squared term. The paper’s stated insight is that controlling ASF through timely departures can reduce extreme fuel consumption and CO$$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$8 emissions. More broadly, this example shows how SQR can support comparative distributional explanation by contrasting formulas learned at central and extreme quantiles.

6. Related uses of the acronym “SQR” and terminological scope

The acronym “SQR” is not unique to Symbolic Quantile Regression. In "A Quantile Regression Approach for Remaining Useful Life Estimation with State Space Models" (Frizzo et al., 20 Jun 2025), “SQR” denotes Simoultaneous Quantile Regression, a distinct construction in which a Structured State Space Model is conditioned on quantile levels and trained end-to-end using an aggregated pinball-loss objective over multiple quantiles.

That work defines, for a fixed quantile grid $$L_\tau(y,f)\;=\;\max\{\tau\,(y-f),\,(\tau-1)\,(y-f)\}.$$9, the aggregate loss

$f$0

and applies it to Remaining Useful Life estimation with State Space Models. The underlying modeling object is therefore a quantile-conditioned sequence model rather than a symbolic expression.

This terminological overlap is relevant because both methods are quantile-regression frameworks using pinball loss, yet they pursue different forms of interpretability and uncertainty representation. Symbolic Quantile Regression seeks human-readable formulas for a chosen $f$1. Simoultaneous Quantile Regression, in the cited State Space Model setting, seeks joint multi-quantile estimation within a learned latent dynamical system. A plausible implication is that “SQR” in contemporary literature should always be disambiguated by expansion or context, especially in cross-disciplinary reading.

7. Strengths, limitations, and research directions

The reported strengths of Symbolic Quantile Regression are threefold: it generates human-readable formulas for any quantile $f$2; it matches or outperforms other transparent methods in predictive accuracy while maintaining very low complexity; and it provides actionable insight into feature effects at different parts of the distribution (Hoekstra et al., 11 Aug 2025). These properties are particularly relevant in high-stakes settings where tail behavior matters and where model transparency is not optional.

The limitations identified in the paper are equally explicit. First, the complexity metric is a heuristic proxy for interpretability. Second, computational cost is higher than that of standard models, although the paper notes mitigation through subsampling or optimized C/Julia implementations. These are structural rather than incidental limitations: the first concerns the operationalization of interpretability, and the second reflects the search-intensive nature of symbolic modeling.

The paper lists three future directions. One is to learn or adapt complexity metrics with human-in-the-loop feedback for domain-specific interpretability. A second is to extend the framework to calibrated prediction intervals, for example via conformal methods, or to joint interval quantile optimization. A third is to investigate robustness under distributional shift and uncertainty quantification in tails.

Taken together, these directions indicate that SQR is best understood as a distribution-aware, white-box modeling framework rather than only as a variant of symbolic regression. Its central contribution is to couple quantile-specific loss minimization with analytic formula discovery, thereby making conditional tail behavior inspectable at the level of explicit expressions rather than only through post hoc explanation.