Expectile Regression Models

• Expectile regression models are statistical methods that minimize an asymmetric quadratic loss to estimate conditional expectiles over the entire response distribution.
• Their differentiable loss function enables efficient computation via techniques like iteratively reweighted least squares, majorization-minimization, and kernel methods.
• Applications include financial risk management, public health, and machine learning, providing robust insights into both central trends and extreme values.

Expectile regression models constitute a class of statistical and machine learning techniques for modeling the conditional expectiles of a response variable, providing a quantile-like characterization of the entire distribution but based on asymmetrically weighted squared loss. Unlike quantile regression, which is rooted in absolute loss, expectile regression minimizes an asymmetric quadratic loss that not only includes the mean (at the 0.5 expectile) but efficiently targets both central and extreme regions of a distribution. Due to the differentiability of the loss function, expectile methods offer computational and inferential advantages in complex real-world settings, such as the modeling of nonlinear, spatial, random, or high-dimensional effects, and have been extended to Bayesian, semiparametric, kernel-based, robust, and high-dimensional frameworks.

1. Fundamental Principles and Mathematical Formulation

Expectiles were introduced as solutions to asymmetrically weighted least squares problems: for a random variable $Y$ and asymmetry parameter $\tau \in (0,1)$, the $\tau$-expectile $e_\tau$ solves

$$e_\tau = \arg\min_m~ E \left[ \tau\, (Y - m)^2 \mathbb{I}_{Y \geq m} + (1-\tau)\, (Y - m)^2 \mathbb{I}_{Y < m} \right].$$

In regression, the conditional expectile function $m(x)$ for covariate $x$ solves

$$m_\tau(x) = \arg\min_m~ E \left[ \tau\, (Y - m)^2 \mathbb{I}_{Y \geq m} + (1-\tau)\, (Y - m)^2 \mathbb{I}_{Y < m}~ |~ X = x \right].$$

The key loss function, often denoted $\rho_\tau(u)$, is

$$\rho_\tau(u) = \begin{cases} \tau\, u^2 & u \ge 0 \ (1-\tau)\, u^2 & u < 0 \end{cases}$$

which results in a smooth, convex objective and enables efficient computation via iterated weighted least squares or related optimization schemes.

A salient feature is the inclusion of the mean as a special case ($\tau = 0.5$); for $\tau \ne 0.5$, expectiles describe asymmetric aspects of the response, resembling but not coinciding with quantiles.

2. Modeling Frameworks and Extensions

2.1 Bayesian and Semiparametric Approaches

A Bayesian formulation aligns the likelihood with the expectile loss by adopting the asymmetric normal distribution. The likelihood for observation $y_i$ centers on the regression function $\eta_i$ and scales residuals according to $\tau$, such that

$$p(y_i) \propto \exp\left\{ -\frac{1}{2\sigma^2} w_\tau(y_i, \eta_i) (y_i - \eta_i)^2 \right\},$$

with weights $w_\tau(\cdot)$ encoding the asymmetry. This likelihood enables full-posterior inference and accommodates diverse model components:

• Linear effects: enter as fixed effects.
• Nonlinear effects: represented via spline or basis function expansions.
• Spatial effects: incorporated using Markov random field structures.
• Random effects: modeled via basis representations with appropriate priors.

Estimation is performed by Markov Chain Monte Carlo with proposal distributions informed by penalized iteratively weighted least squares, enabling efficient posterior sampling and uncertainty quantification (Waldmann et al., 2013).

2.2 Kernel and SVM-type Methods

Flexible nonparametric expectile regression is achievable via reproducing kernel Hilbert spaces (RKHS). In such models, the regression function $f$ is sought in an RKHS $\mathcal{H}_K$:

$$\min_{f \in \mathcal{H}_K,~ \alpha_0 \in \mathbb{R}} \sum_i \rho_\tau(y_i - \alpha_0 - f(x_i)) + \lambda \|f\|^2_{\mathcal{H}_K}$$

where the representer theorem yields a finite expansion of $f$ in terms of kernel evaluations at training points. Efficient majorization-minimization algorithms with provable linear convergence solve these problems, while the theoretical properties offer consistency and asymptotic control (Yang et al., 2015).

Further, support vector machine-like (SVM-like) formulations replace the standard SVM loss with an asymmetric quadratic loss and solve the resulting convex optimization problem efficiently using sequential minimal optimization, with empirical gains over boosting-based ER competitors (Farooq et al., 2015).

2.3 High-dimensional, Robust, and Composite Methods

Recent work addresses high-dimensional and heteroscedastic settings by introducing regularized expectile regression—combining (possibly non-convex) penalties such as SCAD or MCP with expectile loss. These frameworks allow both sparsity and nonlinear effects via partially linear additive models and can handle only finite-moment, heavy-tailed errors (Zhao et al., 2019, Zhao et al., 2019, Man et al., 2022).

Robust expectile regression replaces the standard quadratic loss in each tail with piecewise (Huber-type) losses and applies iteratively reweighted $\ell_1$-penalization (possibly using semismooth Newton methods), yielding oracle properties even in high-dimensional scenarios (Man et al., 2022).

Composite expectile regression combines several expectile levels into a unified objective, borrowing strength across the conditional distribution and improving estimation and variable selection, especially when error distributions are heteroscedastic (Lin et al., 2022).

3. Statistical Inference and Risk Measurement

Expectiles provide a basis for coherent risk measures—unlike quantiles, they satisfy cash-invariance, positive homogeneity, monotonicity, and subadditivity when $\tau \ge 0.5$. The associated statistical functionals are (quasi-)Hadamard differentiable and continuous in the 1-weak topology, so plug-in estimators based on empirical or parametric models are consistent, asymptotically normal, and yield valid bootstrap inference (Krätschmer et al., 2016).

Explicit formulas specify the expectile as a minimizer of a weighted squared deviation, e.g.

$$P_a(X) = \arg \min_m \left\{ a\, \mathbb{E}[(X-m)_+^2] + (1-a)\,\mathbb{E}[(m-X)_+^2] \right\}$$

with detailed derivatives for use in the functional delta method for limiting distribution results.

Recent work has extended expectile methodology to coherent multivariate risk measures, set-valued risk functionals, and law-invariant depths for multivariate outputs (Daouia et al., 2019).

4. Algorithmic and Computational Advances

Efficient estimation techniques for expectile regression exploit the smoothness of the loss function:

• Iteratively Reweighted Least Squares (IRLS): Core to both classical and Bayesian architectures, powering fast estimation with convergence guarantees.
• Majorization-Minimization and Proximal Gradient: Used for kernel methods and high-dimensional penalized problems (Yang et al., 2015, Zhao et al., 2019).
• Sequential Minimal Optimization and Working Set Selection: For SVM-like expectile regression, these yield scalable solvers for large samples (Farooq et al., 2015).
• Coordinate Descent and Local Linear Approximation: Facilitate convex approximations within nonconvex, sparsity-promoting frameworks (SCAD/MCP) (Man et al., 2022).
• EM Algorithm: Used in hidden Markov expectile models for efficient likelihood-based inference (Foroni et al., 2023).

Recent innovations include robustification through Huber-type modifications and the use of adaptive loss functions to counteract the impact of heavy tails and outliers in modern data regimes (Man et al., 2022, Zhao et al., 2019).

5. Applications and Empirical Validation

Applications of expectile regression models are widespread:

• Financial risk management: Expectiles directly underpin expectile-based Value-at-Risk (EVaR) and Expected Shortfall estimation. They offer superior tail sensitivity and calibration of risk measures relative to quantile-based VaR, especially during volatile and crisis periods. Dynamic, regime-switching, and multivariate expectile models have been shown to outperform traditional models in forecasting and backtesting on major indices (Oketunji, 16 Jul 2025, Wang et al., 2019). Extended backtesting procedures confirm improved model risk and predictive performance.
• Public health, genetics, and epidemiology: Expectile regression uncovers heterogeneous effects in health outcomes (e.g., childhood malnutrition), enables full-distribution mapping in phenotypic risk, and, when paired with neural networks, captures nonlinear and gene-gene interactions otherwise missed by linear methods (Lin et al., 2020, Waldmann et al., 2013).
• Electricity price forecasting: Expectile regression averaging delivers probabilistic forecasts with improved coverage and accuracy over quantile regression, especially after variance-stabilizing transformations (e.g., via the asinh function), as demonstrated for day-ahead electricity prices in Germany (Janczura, 12 Feb 2024).
• Dimension reduction and machine learning: Expectile-assisted inverse regression enables sufficient dimension reduction robust to heteroscedasticity, outperforming traditional moment-based methods in both simulation and real data (Soale et al., 2019). In time series, expectile periodograms offer a two-dimensional spectral analysis, with superior capability to detect hidden periodicities and strong empirical performance in deep learning-based seismic signal classification (Chen, 4 Mar 2024).
• Matrix factorization and network latency: Expectile matrix factorization generalizes conventional low-rank approaches for skewed or heavy-tailed patterns, providing more robust and informative recovery for web service latency and recommender systems (Zhu et al., 2016).

6. Model Selection, Extensions, and Theory

Model selection in expectile regression may leverage hierarchical or composite objectives, $L_1$-type (lasso) or folded concave penalties (SCAD, MCP), and can target multiple expectile levels simultaneously. Efficient coordinate descent or EM algorithms enable high-dimensional and complex settings (Lin et al., 2022).

In panel and longitudinal data, expectile regression with fixed effects (ERFE) extends the within transformation to estimate conditional expectiles free from bias due to omitted time-invariant effects. ERFE provides consistent estimation, robust standard errors, and scalable software for large panels (Barry et al., 2021).

Theoretical advances guarantee oracle properties for sparse recovery under mild conditions, nonasymptotic error bounds, and strong inferential guarantees in both standard and heavy-tailed environments (Man et al., 2022, Zhao et al., 2019). Asymptotic and finite sample properties have been rigorously developed for both frequentist and Bayesian frameworks, and for a variety of data types—including nonparametric, parametric, and high-dimensional designs.

7. Future Directions and Implications

Active research directions for expectile regression include:

• Deeper integration of robust and adaptive loss functions for handling non-subgaussian errors and extreme events.
• Further extension to multivariate, functional, or network-valued responses, with corresponding generalizations of risk measures and centrality concepts.
• Adoption in real-time and online machine learning frameworks (e.g., risk-averse Bayesian optimization with variational inference and batch query strategies (Picheny et al., 2020)).
• Broader application across domains encountering heavy-tailed, heteroscedastic, or nonstationary data.

A plausible implication is that expectile-based regression and risk frameworks are poised to play a central role in next-generation risk management, high-dimensional inference, and interpretable machine learning due to their coherence, computational tractability, and ability to span the full distributional range of the response variable.