Upper Expectile Regression

• Upper expectile regression is a statistical method that models conditional upper-tail extremes by minimizing an asymmetrically weighted squared loss.
• The Bayesian geoadditive framework employs an auxiliary asymmetric normal likelihood to seamlessly integrate linear, nonlinear, spatial, and random effects.
• Efficient MCMC algorithms with blockwise IWLS proposals enable robust posterior inference and full uncertainty quantification for tail-specific estimates.

Upper expectile regression is a statistical modeling strategy that characterizes the conditional upper tail of a response distribution by minimizing an asymmetrically weighted squared loss, making it an alternative to quantile regression for modeling extremes. In the Bayesian geoadditive context, this framework extends the model's flexibility to accommodate linear, nonlinear, spatial, and random effects via an auxiliary asymmetric normal distribution as likelihood, enabling full inference and semiparametric specification (Waldmann et al., 2013).

1. Definition and Foundation of Upper Expectile Regression

Upper expectile regression differs from quantile regression by employing an asymmetrically weighted squared loss rather than the check (absolute) loss. For a given asymmetry parameter $\tau \in (0,1)$, the expectile $\eta_{i,\tau}$ is estimated by minimizing

$$\sum_{i=1}^n w_{(\tau)}(y_i, \eta_{i,\tau}) \cdot (y_i - \eta_{i,\tau})^2,$$

where $w_{(\tau)}(y_i, \eta_{i,\tau}) = \tau$ if $y_i \geq \eta_{i,\tau}$ and $1-\tau$ otherwise. The estimator emphasizes the upper tail as $\tau \to 1$, thus characterizing upper extremes.

The estimation is computationally efficient due to the differentiability of the squared loss, accommodating solution via iteratively weighted least squares (IWLS). Unlike quantiles, expectiles do not have a cumulative distribution interpretation but represent a form of data-weighted tail expectation.

2. Bayesian Semiparametric Framework and Auxiliary Asymmetric Normal

The Bayesian geoadditive expectile regression model (Waldmann et al., 2013) adopts an auxiliary Asymmetric Normal Distribution (AND) as likelihood:

$y_i \sim AN(\eta_i, \sigma^2, \tau),$

with kernel

$$p(y_i) \propto \exp\Big\{-\frac{1}{2\sigma^2} w_{(\tau)}(y_i, \eta_{i,\tau}) (y_i - \eta_{i,\tau})^2\Big\}.$$

This induces the same point estimates as the classical frequentist expectile regression but enables full Bayesian inference, including posterior credible intervals and straightforward integration of prior knowledge.

The predictor is formulated as $\eta_i = \beta_0 + \sum_j f_j(z_i)$, where each function $f_j$ may be a linear effect, penalized spline (nonlinear effect), spatial Markov random field, or random effect. Each $f_j$ is expressed as a basis expansion $f_j(z) = \sum_k \beta_{jk} B_k(z)$, allowing additive decomposition and modular prior/penalty structures.

Priors are specified as multivariate normal (with smoothing penalties) for coefficients, and inverse gamma for variance and smoothing parameters. The hierarchical model fully accommodates mixed-effect, spatial, and nonparametric features.

3. MCMC Algorithm Leveraging Iteratively Weighted Least Squares

Posterior inference is performed via a Markov chain Monte Carlo (MCMC) algorithm using proposal densities inspired by the frequentist IWLS update step. For component $j$,

• Propose $\beta_j' \sim N(\mu_j, \Sigma_j)$ with

$\mu_j = \Sigma_j B_j^T W_{(\tau)} (y - \eta_{-j}),$

$$\Sigma_j = (B_j^T W_{(\tau)} B_j + \lambda_j K_j)^{-1},$$

where $W_{(\tau)}$ is a diagonal matrix of weights depending on the current state, $\eta_{-j}$ excludes the $j$th component, and $\lambda_j = \sigma^2/\delta_j^2$ is a smoothing parameter for component $j$.

This strategy enables efficient traversal of the high-dimensional posterior, even for complex, misspecified (auxiliary) likelihoods required by expectile regression. Each full conditional for $\beta_j$ is updated in block Gibbs or Metropolis-Hastings, with smoothing and prior parameters updated by standard Bayesian steps (e.g., gamma random variables for variances).

4. Unified Modeling of Linear, Nonlinear, Spatial, and Random Effects

A salient feature is the model's semiparametric additivity:

• Linear effects are handled as unpenalized basis expansions.
• Nonlinear effects use penalized B-splines (cubic, $B$-splines with penalty matrix $D_k^T D_k$), with smoothness enforced by the structure of $K_j$.
• Spatial effects employ Markov random fields, with region-wise indicator bases and spatial adjacency matrices as penalty.
• Random effects exploit the same machinery as splines, with proper priors.

Each effect can be flexibly switched on or off, leading to high modularity and adaptability to various data structures (including spatially correlated and hierarchical data).

5. Tail Behavior, Inference, and Real-World Applications

Upper expectile regression (large $\tau$) directly targets the distribution's upper extremities, offering point and interval estimates specific to high-outcome behavior. In the childhood malnutrition application, expectile regression revealed more pronounced or qualitatively different effects for $\tau=0.95$ compared to the mean, e.g., spatial heterogeneity and the impact of socioeconomic covariates.

The approach supports:

• Noncrossing expectile curves due to the differentiable quadratic loss,
• Inference in the tails—credible intervals for extreme quantiles under model-based uncertainty,
• Decomposition of covariate effects by distributional location—critical for risk stratification and high-risk subgroup identification.

Performance is robust to high model complexity due to the penalized, modular nature of the additive predictor and the auxiliary likelihood.

6. Implementation Considerations and Computational Trade-Offs

Key implementation considerations include:

• Computational demand increases with the number of basis functions and covariate interactions; however, using blockwise IWLS proposals and efficient MCMC updating schemes (incorporating prior structure) allows scaling to real datasets with nonlinear and spatial complexity.
• Choice of smoothing/prior parameters directly impacts overfitting and the bias-variance trade-off, and should be addressed via model selection or hierarchical Bayesian learning.
• Posterior summarization yields not only point estimates but full credible intervals and uncertainty assessments for all model components, crucial for scientific interpretation.

7. Extensions and Impact

The Bayesian geoadditive expectile regression framework unifies classical frequentist semiparametric modeling with full posterior inference, generalizing naturally to multiple response types, higher-dimensional spatial structures, and scenarios requiring tail modeling (risk management, epidemiology, environmental statistics). Compared with quantile regression, the differentiable squared loss facilitates estimation and inference, and modeling of the upper tails via large $\tau$ is direct.

The method accommodates both Bayesian hierarchical learning of complex effect structures and computation of tail-focused effects that are otherwise inaccessible via mean-centric or standard quantile regression, supporting applications where modeling of extremes is imperative.

---

Summary Table: Core Features of Bayesian Geoadditive Upper Expectile Regression

Feature	Description	Implementation Mechanism
Loss Function	Asymmetrically weighted squared loss	Auxiliary asymmetric normal likelihood
Effect Types	Linear, nonlinear, spatial, random	Basis expansion + penalty/prior struct.
Inference	Full Bayesian, tail-specific intervals	MCMC, IWLS-based proposals
Tail Focus	Direct modeling of upper tails ($\tau\to1$)	Modulate loss weights
Computational Efficiency	Fast for moderate complexity, modular updates	Blockwise IWLS in MCMC
Real-world Applicability	Risk stratification, heteroscedastic, spatial data	Additive predictor, spatial MRF

This approach provides a rigorous and flexible platform for modeling and inference in upper tail regression analyses, especially when the system under paper requires granular quantification of extreme (adverse or high-benefit) outcomes.