---
title: Beta Regression in High Dimensions
url: https://www.emergentmind.com/topics/beta-regression-with-high-dimensional-predictors
type: topic
---

# Beta Regression in High Dimensions

Beta regression with high-dimensional predictors is the extension of beta regression to settings in which the response is a bounded continuous variable in \((0,1)\)—such as a proportion, ratio, rate, or bounded score—while the predictor dimension is large relative to the sample size and regularization or structured shrinkage is required for stable estimation. In this regime, standard linear regression is inappropriate because it can produce predictions outside \((0,1)\) and does not respect the bounded, often skewed, nature of the data. Recent work organizes the area around three main strategies: \(\ell_1\)-penalized frequentist estimation with non-asymptotic theory and debiasing [2507.20079], Bayesian beta regression with a Horseshoe prior and fractional posterior [2505.22211], and ridge-type shrinkage estimators that target multicollinearity, weak predictors, and unstable maximum likelihood estimation [2111.13887].

## 1. Canonical model and high-dimensional formulation

The common starting point is the mean/precision parameterization of the beta distribution. For \(Y_i \in (0,1)\), the conditional law is written as
\[
Y_i \mid X_i \sim \mathrm{Beta}(\mu_i,\phi_i),
\]
or equivalently,
\[
y_i \sim \mathrm{Beta}(\mu_i \phi, (1-\mu_i)\phi),
\]
with density
\[
f(y;\mu,\phi) = \frac{\Gamma(\phi)} {\Gamma(\mu\phi)\Gamma((1-\mu)\phi)} y^{\mu\phi-1}(1-y)^{(1-\mu)\phi-1}, \qquad y\in(0,1).
\]
Across the recent literature, the first two moments are
\[
\mathbb{E}(Y_i)=\mu_i, \qquad \mathrm{Var}(Y_i\mid X_i)=\frac{\mu_i(1-\mu_i)}{1+\phi}.
\]
The mean is typically linked to predictors by the logit specification
\[
\mu_i=\mathrm{logit}^{-1}(X_i^\top\beta)=\frac{1}{1+e^{-X_i^\top\beta}},
\]
or, with an intercept,
\[
\mu_i = \frac{\exp(\beta_0 + X_i^\top \beta)}{1+\exp(\beta_0 + X_i^\top \beta)}.
\]

What makes the setting high-dimensional is not the beta likelihood itself but the regime in which \(p\) may exceed \(n\), or more strongly \(p \gg n\), together with a sparsity assumption on the true regression vector. The Bayesian paper states this explicitly as
\[
s^* := \|\beta_0\|_0 < n < p,
\]
while the frequentist LASSO paper treats the goal as estimation of a sparse \(\beta^*\) when \(p\) is large [2505.22211; 2507.20079]. The ridge-type paper frames the high-dimensional case through a large number of inactive or weak predictors, often combined with severe predictor correlation [2111.13887].

The role of the precision parameter \(\phi\) differs across methods. The LASSO analysis assumes a homogeneous scale parameter,
\[
\phi_i \equiv \phi^*,
\]
for simplicity, though it notes that varying dispersion is possible in principle. The Bayesian development treats \(\phi\) as fixed in the theoretical development and simulation study. The ridge-type paper works with joint likelihood-based inference for \((\beta,\phi)\) and writes the Fisher information in block form [2507.20079; 2505.22211; 2111.13887].

| Paper | High-dimensional mechanism | Distinctive feature |
| --- | --- | --- |
| "Lasso Penalization for High-Dimensional Beta Regression Models: Computation, Analysis, and Inference" [2507.20079] | \(\ell_1\)-penalty on \(\beta\) | stationary-point theory, debiasing, proximal gradient |
| "Handling bounded response in high dimensions: a Horseshoe prior Bayesian Beta regression approach" [2505.22211] | Horseshoe prior and fractional posterior | Polya–Gamma Gibbs sampler, posterior consistency |
| "Ridge-Type Shrinkage Estimators in Low and High Dimensional Beta Regression Model with Application in Econometrics and Medicine" [2111.13887] | ridge stabilization and shrinkage toward a restricted model | closed-form bias and variance, multicollinearity focus |

## 2. \(\ell_1\)-penalized beta regression and nonconvex high-dimensional theory

The frequentist high-dimensional formulation in [2507.20079] minimizes an \(\ell_1\)-penalized beta-regression objective. Writing \(\theta=(\beta_0,\beta,\phi)\), the negative log-likelihood is
\[
R_n(\theta) = -\log \Gamma(\phi) + \frac{1}{n}\sum_{i=1}^n \Big[ \log \Gamma(\mu_i\phi) + \log \Gamma((1-\mu_i)\phi) - (\mu_i\phi)\log Y_i - ((1-\mu_i)\phi-1)\log(1-Y_i) \Big],
\]
with
\[
\mu_i = \frac{\exp(\beta_0 + X_i^\top \beta)}{1+\exp(\beta_0 + X_i^\top \beta)}.
\]
The estimator is defined by
\[
\widehat{\theta} \in \arg\min_\theta \left\{ R_n(\theta)+\lambda_n \|\beta\|_1 \right\}.
\]
Only \(\beta\) is penalized; the intercept \(\beta_0\) and precision \(\phi\) are not penalized.

The technical difficulty is that the beta-regression negative log-likelihood is not globally convex in the regression parameters. A second complication is that the gradient involves \(\log(Y_i)\) and \(\log(1-Y_i)\), which are unbounded near \(0\) and \(1\). Since beta random variables can get arbitrarily close to the endpoints, the analysis requires truncation and careful tail bounds. To handle this, the paper uses the framework of Elsener and van de Geer (2018) for stationary points of nonconvex penalized M-estimators in a neighborhood of the target parameter [2507.20079].

The theory is local. Its main assumptions are a sub-Gaussian design,
\[
\sup_{\|v\|_2\le 1}\|X^\top v\|_{\psi_2} < C_X < \infty,
\]
local strong convexity of the population risk at \(\beta^*\),
\[
\min_{\|v\|_2=1} v^\top \nabla^2 R(\beta^*) v \ge \underline{\mu}>0,
\]
and bounded linear predictors in a neighborhood \(\mathcal B\) of \(\beta^*\),
\[
|X^\top \beta| \le h \quad \text{a.s.}
\]
This last condition implies
\[
c_h \le \mu_i(\beta) \le 1-c_h, \qquad c_h = \frac{1}{1+e^h},
\]
which keeps the fitted means away from the boundaries and is used to control beta-function and digamma terms.

A technical lemma establishes local strong convexity: there exists a radius \(\eta\) such that for all
\[
\beta \in \mathcal B = \{\beta : \|\beta-\beta^*\|_2\le \eta\},
\]
the Hessian satisfies
\[
\min_{\beta\in\mathcal B} \min_{\|v\|_2=1} v^\top \nabla^2 R(\beta)v \ge \underline{\mu}/2.
\]
The principal result is then a non-asymptotic \(\ell_1\)-error bound for any stationary point in \(\mathcal B\). If \(\lambda_n\asymp \lambda_0\asymp \log(n)\sqrt{\log(p)/n}\), the paper obtains
\[
\|\widehat{\beta}-\beta^*\|_1 = O\!\left(s\,\log(n)\sqrt{\frac{\log p}{n}}\right),
\]
so consistency holds if
\[
s^2 \log^2(n)\log(p)/n \to 0.
\]
The paper explicitly notes that the extra \(\log^2(n)\) factor, relative to some GLM results, arises from the sub-exponential tails of \(\log Y\) and \(\log(1-Y)\), and that this may or may not be improvable [2507.20079].

## 3. Debiasing, confidence intervals, and proximal-gradient computation

The same frequentist framework also develops inference for individual coefficients. Starting from the KKT conditions,
\[
\nabla_{\beta_0,\beta}R_n(\widehat{\theta}) + \lambda_n \widehat z = 0,
\]
with \(\widehat z\) a subgradient vector with entries in \([-1,1]\), a Taylor expansion around the true parameter motivates a debiasing correction using an approximate inverse Hessian \(\widehat\Omega\). The debiased estimator is
\[
\begin{pmatrix} \widehat\beta_0^{\mathrm{db}} \\ \widehat\beta^{\mathrm{db}} \end{pmatrix}
=
\begin{pmatrix} \widehat\beta_0 \\ \widehat\beta \end{pmatrix}
+ \lambda_n \widehat\Omega \widehat z,
\]
where \(\widehat\Omega\) is chosen so that
\[
\left\| \nabla^2_{\beta_0,\beta} R_n(\widehat\theta)\,\widehat\Omega - I_{p+1} \right\|_\infty \lesssim \lambda_0.
\]
The covariance estimator is
\[
\frac{1}{n}\widehat\Omega \widehat M \widehat\Omega^\top, \qquad \widehat M = \sum_{i=1}^n \nabla_{\beta_0,\beta} R_n^i(\widehat\theta)^{\otimes 2}.
\]
Under suitable conditions, the debiased estimator is asymptotically normal and coordinate-wise confidence intervals can be constructed from the diagonal entries of this covariance estimator. The simulations indicate that coverage is reasonably close to nominal in the moderate-dimensional regime, but deteriorates when \(s\) and \(p\) grow, reflecting the stronger conditions required for valid debiasing relative to estimation consistency [2507.20079].

For optimization, the paper proposes an alternating scheme consisting of proximal gradient descent on \((\beta_0,\beta)\) with \(\phi\) fixed, followed by one-dimensional minimization or coordinate update for \(\phi\) with \((\beta_0,\beta)\) fixed. The proximal step is
\[
\beta_0' = \beta_0^{(t)} - s\, \nabla_{\beta_0}R_n(\theta^{(t)}),
\]
\[
\beta' = \mathcal S_{\lambda_n s}\!\left(\beta^{(t)} - s\,\nabla_\beta R_n(\theta^{(t)})\right),
\]
where the soft-thresholding operator is
\[
\mathcal S_\tau(z)_j = \mathrm{sign}(z_j)\max\{|z_j|-\tau,0\}.
\]
A backtracking rule reduces the stepsize when the local quadratic majorization condition fails:
\[
R_n(\theta') \le R_n(\theta^{(t)}) + \nabla_{\beta_0,\beta}R_n(\theta^{(t)})^\top(\theta'-\theta^{(t)}) + \frac{1}{2s}\|\theta'-\theta^{(t)}\|_2^2.
\]
The algorithm stops when the objective decrease is below tolerance. Because the objective is nonconvex, global optimality is not guaranteed; the theoretical guarantee applies when iterates remain in the local convex neighborhood around \(\beta^*\) [2507.20079].

## 4. Bayesian Horseshoe beta regression

The Bayesian alternative in [2505.22211] is explicitly designed for the regime
\[
p \gg n,
\]
under a sparse truth. The observation model is
\[
y_i \mid X_i,\beta \sim \mathrm{Beta}(\mu_i\phi, (1-\mu_i)\phi), \qquad \mu_i=\mathrm{logit}^{-1}(X_i^\top\beta),
\]
with likelihood
\[
L_n(\beta) = \prod_{i=1}^n \left[ \frac{\Gamma(\phi)}{\Gamma(\mu_i \phi)\Gamma((1-\mu_i)\phi)} y_i^{\mu_i\phi-1}(1-y_i)^{(1-\mu_i)\phi-1} \right].
\]
The prior is the Horseshoe:
\[
\beta_j \mid \lambda_j,\tau \sim \mathcal{N}(0,\lambda_j^2\tau^2),  \qquad \lambda_j \sim \mathrm{Cau}_+(0,1), \qquad \tau \sim \mathrm{Cau}_+(0,1), \quad j=1,\dots,p.
\]
This yields global shrinkage through \(\tau\) and local shrinkage through \(\lambda_j\), with strong shrinkage toward zero for noise predictors and heavy tails for large signals.

For Gibbs sampling, the paper uses the Makalic–Schmidt inverse-gamma representation,
\[
\lambda_j^2 \mid \nu_j \sim \mathrm{Inv\text{-}Gamma}\left(\frac12,\frac{1}{\nu_j}\right), \qquad \nu_j \sim \mathrm{Inv\text{-}Gamma}\left(\frac12,1\right),
\]
\[
\tau^2 \mid \xi \sim \mathrm{Inv\text{-}Gamma}\left(\frac12,\frac{1}{\xi}\right), \qquad \xi \sim \mathrm{Inv\text{-}Gamma}\left(\frac12,1\right).
\]
The posterior is tempered:
\[
\pi_{n,\alpha}(\beta) \propto L_n(\beta)^\alpha \pi(\beta), \qquad \alpha \in (0,1).
\]
The paper treats this as a fractional posterior. It states that values like \(\alpha=0.99\) are practically close to standard Bayes while preserving theoretical benefits.

A major computational contribution is a Polya–Gamma augmentation scheme that avoids Metropolis–Hastings steps. Defining
\[
\kappa_i = \phi\left(y_i-\frac12\right), \qquad \eta_i = X_i^\top\beta,
\]
and latent variables \(\omega_i \sim \mathrm{PG}(\phi,\eta_i)\), the augmented likelihood becomes conditionally Gaussian:
\[
\log p(y_i\mid \eta_i) \propto \kappa_i \eta_i - \frac{\omega_i \eta_i^2}{2},
\]
\[
p(\mathbf y \mid \beta,\omega) \propto \exp\left( -\frac12 \beta^\top X^\top \Omega X \beta + \beta^\top X^\top \boldsymbol{\kappa} \right),
\]
where \(\Omega=\mathrm{diag}(\omega_1,\dots,\omega_n)\). Given \(\omega,\lambda,\tau\), the conditional posterior is
\[
\beta \mid \omega,\lambda^2,\tau^2,\mathbf y \sim \mathcal{N}(m,V),
\]
with
\[
V = \left(X^\top \Omega X + \Lambda^{-1}\tau^{-2}\right)^{-1}, \qquad m = V X^\top \boldsymbol{\kappa}.
\]
The remaining Gibbs steps update the local scales, auxiliary local hyperparameters, global scale, and global auxiliary hyperparameter in closed form.

The theoretical contribution is the first set of posterior concentration guarantees for Bayesian beta regression in this setting. Under assumptions including
\[
p \le \exp(n^b), \qquad b<1,
\]
\[
\|\beta_0\|_\infty \le C_1,
\]
\[
\mathbb{E}\|X\|^2 \le C_x < \infty,
\]
and
\[
|X^\top \beta_0| \le C_2 < \infty \quad \text{a.s.},
\]
the paper proves concentration of the fractional posterior in Rényi divergence at rate
\[
\varepsilon_n = K\,\frac{s^*\log(p/s^*)}{n}.
\]
It further derives corresponding bounds for Hellinger distance, total variation distance, and the linear predictor error
\[
\|X^\top(\beta-\beta_0)\|_2^2,
\]
and states that the posterior mean estimator satisfies the same rate. Practical implementation is provided by the R package `betaregbayes`, run in the paper with 1200 iterations, 200 burn-in iterations discarded, and posterior summaries computed from posterior draws [2505.22211].

## 5. Ridge-type shrinkage under multicollinearity and weak predictors

A different line of work emphasizes that high-dimensional beta regression is often difficult not only because \(p\) is large, but because the Fisher information matrix can become ill-conditioned when predictors are highly correlated. The ridge-type framework in [2111.13887] targets three problems simultaneously: multicollinearity, weak or insignificant predictors, and unstable MLEs obtained from a nonlinear likelihood.

The model remains
\[
y_i \sim \text{Beta}(\mu_i\phi,(1-\mu_i)\phi),\qquad g(\mu_i)=x_i^\top\beta,
\]
usually with the logit link
\[
\mu_i=\frac{\exp(x_i^\top\beta)}{1+\exp(x_i^\top\beta)}.
\]
The unrestricted ridge estimator is written as
\[
\widehat{\beta}^{UR} = \left(X^\prime \widehat{\mu} X + k I_p\right)^{-1}X^\prime \widehat{\mu} z,
\]
with ridge parameter
\[
\widehat{k}=\frac{1}{\widehat{\varphi}^\prime \widehat{\varphi}},
\]
where \(\widehat{\varphi}=C^\prime\widehat{\beta}\) and \(C\) contains eigenvectors of \(X^\prime \widehat{\mu}X\).

The paper then introduces a restricted estimator under the linear restriction
\[
H_0: H\beta=h,
\]
namely
\[
\widehat{\beta}^{R} = \widehat{\beta} - \mathcal{I}^{-1}H^\prime \left(H\mathcal{I}^{-1}H^\prime\right)^{-1} (H\widehat{\beta}-h),
\]
and combines unrestricted and restricted fits through several shrinkage rules. Representative examples are the ridge-type linear shrinkage estimator
\[
\widehat{\beta}^{RLS} = \delta\,\widehat{\beta}^{RR} + (1-\delta)\,\widehat{\beta}^{UR}, \qquad 0\le \delta \le 1,
\]
the ridge-type pretest estimator
\[
\widehat{\beta}^{RPT} = \widehat{\beta}^{UR} - (\widehat{\beta}^{UR}-\widehat{\beta}^{RR})\,I(T_n\le T_{n,\alpha}),
\]
the ridge-type Stein estimator
\[
\widehat{\beta}^{RS} = \widehat{\beta}^{RR} + \left[1-\frac{p_2-2}{T_n}\right](\widehat{\beta}^{UR}-\widehat{\beta}^{RR}),
\]
and the ridge-type positive Stein estimator
\[
\widehat{\beta}^{RPS} = \widehat{\beta}^{RR} + \left[1-\frac{p_2-2}{T_n}\right]^+ (\widehat{\beta}^{UR}-\widehat{\beta}^{RR}).
\]

These estimators are not standard ridge estimators that shrink directly toward zero. Rather, they shrink between a full model and a restricted model. This distinction is central in the paper’s interpretation: a hard variable-selection procedure discards weak predictors entirely, whereas the ridge-type estimators attempt to recover as much information as possible from weak predictors while still borrowing strength from a reduced model. Their asymptotic analysis is carried out under local alternatives,
\[
\mathcal{K}_{(n)}:\quad H\beta=h+\frac{\vartheta}{\sqrt{n}},
\]
and develops closed-form expressions for asymptotic distributional bias and variance. The main analytical conclusions are that restricted and shrinkage estimators can have markedly smaller mean squared error than the unrestricted ridge estimator when the restriction is close to true, while the fully restricted estimator deteriorates when the restriction is wrong; Stein-type and positive Stein-type estimators improve over the unrestricted estimator over a broad parameter range, but no universal dominance is claimed [2111.13887].

## 6. Simulation evidence and empirical applications

The three strands of the literature evaluate performance in distinct but complementary regimes.

**Penalized likelihood and inference.** The LASSO paper studies both a lower-dimensional setting and a genuinely high-dimensional setting with \(p>n\). Its simulations report that \(\ell_1\)-error decreases roughly like \(s/\sqrt{n}\) up to polylog factors, that fitted regressions for \(\log\|\widehat\beta-\beta^*\|_1\) show positive dependence on \(s\), negative dependence on \(n\), and polylog dependence on \(p\), and that LASSO attains low false positive rates and high true positive rates for moderate sparsity, although some extraneous variables are selected. The real-data application uses \(n=2377\) U.S. counties and \(p=20\) centered and scaled covariates related to income, health, crime, education, demographics, and health-service access to model the proportion of individuals incarcerated. The method identifies a sparse set of important predictors including poor physical health days, healthcare costs, police officers, children in poverty, and several demographic or structural variables. These selected variables broadly match those found by an exhaustive AIC search, though the LASSO model is somewhat less parsimonious [2507.20079].

**Bayesian global-local shrinkage.** The Horseshoe paper evaluates low-dimensional settings with \(p=20\), \(s^*=10\), and \(n \in \{100,500,1000\}\), and high-dimensional settings with \(n=80,p=100\) and \(n=200,p=300\), under both independent and correlated designs. Its performance metrics include
\[
\ell_2(\beta_0)=p^{-1}\|\hat\beta-\beta_0\|_2^2,
\]
\[
\ell_2(X^\top\beta_0)=n^{-1}\|X^\top\hat\beta-X^\top\beta_0\|_2^2,
\]
\[
\ell_2(Y)=n^{-1}\sum_{i=1}^n(y_i-\hat\mu_i)^2,
\]
as well as test-set error, precision, recall, specificity, F1, and FDR. The paper reports that Horseshoe matches or outperforms standard beta regression in low dimensions, is much better than transformed Lasso, and in many high-dimensional settings attains precision near 1, specificity near 1, and FDR near 0. Its real-data GPA ratio analysis identifies a single strong negative predictor, “often distracted,” and yields
\[
100\times \ell_2(Y_{\rm test}) = 0.453 \text{ vs } 0.564
\]
for Horseshoe versus beta regression [2505.22211].

**Ridge-type shrinkage under collinearity.** The ridge-type paper generates predictors from a multivariate normal distribution with covariance
\[
\Sigma_{ij}=\rho^{|i-j|},
\]
using \(\rho=0.6\) and \(0.9\) in low dimensions and \(\rho=0.95\) and \(0.99\) in high dimensions, with \(\phi=5\). High-dimensional simulations consider \(n=200\), \(p_1=10\), and \(p_2=500\) or \(1000\). The study finds that increasing correlation increases RMSE for all estimators, larger \(p_2\) increases RMSE, the restricted ridge estimator performs best when \(\Delta=0\), and Stein and positive Stein estimators are uniformly better than the unrestricted estimator over much of the low-dimensional parameter space. In the Dutch city budget application, a condition number
\[
CN(X^\prime \widehat{\mu}X)=809.097
\]
is presented as evidence of strong multicollinearity, and the proposed shrinkage estimators yield smaller bootstrap standard errors than unrestricted MLE. In the body fat application, the paper emphasizes that with \(p=13\) and \(n=10\), ordinary beta regression breaks down; after augmentation with 1000 noise variables, Boruta selects many noise variables and betaboost performs poorly, while the proposed shrinkage estimators again provide smaller standard errors and more stable coefficient estimates [2111.13887].

## 7. Methodological contrasts, recurrent misconceptions, and open issues

A recurring simplification is to treat high-dimensional beta regression as a routine extension of generalized linear modeling. The recent literature instead makes clear that the bounded-response setting introduces distinct technical obstacles: the beta-regression negative log-likelihood is not globally convex, the score can contain unbounded \(\log(Y_i)\) and \(\log(1-Y_i)\) terms, and severe collinearity can destabilize both likelihood-based and restricted estimators [2507.20079; 2111.13887]. The Bayesian work adds a further point: theoretical guarantees are harder to obtain because the model is not in the natural exponential family, which is one reason for the use of a fractional posterior [2505.22211].

The three approaches also differ in what they regularize. The LASSO paper enforces sparsity directly through \(\lambda_n\|\beta\|_1\) and then corrects selected coordinates by debiasing. The Bayesian paper uses global-local shrinkage through the Horseshoe prior and reports posterior concentration at
\[
\varepsilon_n \asymp \frac{s^*\log(p/s^*)}{n}.
\]
The ridge-type paper does not primarily target exact sparsity; it stabilizes estimation under multicollinearity and shrinks toward a restricted model when some coefficients are believed to be zero or negligible. This suggests that method choice depends on whether the main concern is sparse recovery with formal high-dimensional inference, posterior uncertainty quantification under global-local shrinkage, or variance reduction under highly correlated predictors.

Several limitations are explicit in the current literature. In the LASSO framework, global optimality is not guaranteed because the objective is nonconvex, and the extra \(\log^2(n)\) factor in the error rate may or may not be improvable [2507.20079]. In the debiasing stage, coverage deteriorates as dimensionality and sparsity increase. In the Bayesian framework, performance is best near the true precision parameter \(\phi\), though it remains competitive under moderate misspecification [2505.22211]. In the ridge-type framework, the fully restricted estimator deteriorates quickly as \(\Delta\) grows, so the practical advantage lies in adaptive shrinkage estimators rather than hard restriction [2111.13887].

Taken together, these developments define beta regression with high-dimensional predictors as a field organized around bounded-response likelihoods, sparsity or shrinkage under \(p\) large relative to \(n\), and method-specific solutions to nonconvexity, collinearity, and inferential uncertainty. The current state of the literature already supports computation, analysis, and real-data use, but it also leaves open the refinement of rates, the robustness of inferential procedures in more extreme regimes, and the broader treatment of varying dispersion.

Source: https://www.emergentmind.com/topics/beta-regression-with-high-dimensional-predictors