Highly Adaptive Lasso (HAL) Overview

• Highly Adaptive Lasso (HAL) is a nonparametric framework for estimating infinite-dimensional functions with bounded sectional variation using an L1-type regularization.
• It leverages a massive dictionary of indicator and spline bases to adaptively perform regression, density, hazard, and causal inference under minimal smoothness assumptions.
• HAL supports sparse, data-adaptive estimates with strong theoretical guarantees, including near dimension-free convergence rates and plug-in efficiency for semiparametric inference.

Searching arXiv for recent and foundational HAL papers to ground the article. The Highly Adaptive Lasso (HAL) is a nonparametric minimum-loss estimation framework for infinite-dimensional target functions defined over classes of càdlàg functions with bounded sectional variation norm. Its defining construction replaces low-dimensional parametric structure or local smoothness assumptions with a global variation constraint that induces an $L_1$-type regularization on a very large dictionary of lower-orthant indicator bases, and, in higher-order variants, tensor-product spline bases. Across the HAL literature, the method is formulated as an empirical risk minimizer under a sectional variation norm bound, with cross-validation commonly used to select the effective complexity level. This yields a sparse, data-adaptive estimator that has been used for regression, density estimation, hazard estimation, causal nuisance estimation, plug-in estimation of non-pathwise differentiable functionals, and targeted learning, while supporting rates and inferential results that differ substantially from classical kernel- or sieve-based nonparametrics (Laan et al., 2017, Laan, 2023, Munch et al., 2024, Wang et al., 11 Feb 2026).

1. Function class, representation, and optimization principle

HAL targets function classes consisting of multivariate càdlàg functions with bounded sectional variation norm. In one standard formulation, the parameter of interest is a population risk minimizer

$P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$

and the estimator is the empirical risk minimizer over a bounded-variation class,

$$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$

For a $d$-variate càdlàg function on a compact rectangle $[0,\tau]$, the uniform sectional variation norm is written as

$$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$

where $\psi_s(u_s)=\psi(u_s,0_{-s})$ denotes the section indexed by $s$ (Laan et al., 2017).

A central structural fact is that bounded sectional variation induces an integral representation in lower-orthant indicators. For the zero-order setting, one representation used in the literature is

$$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$

which shows that $\psi$ can be written as an infinite linear combination of indicator basis functions $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$0. In regression-style notation, this becomes a discrete basis expansion over observed knot points,

$P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$1

with variation norm

$P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$2

Thus the sectional variation norm is identified with the coefficient $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$3-norm, which is why HAL is “lasso-like” despite being defined over an infinite-dimensional function class (Wang et al., 11 Feb 2026, Laan et al., 2017).

The induced optimization problem is therefore an empirical risk minimization over a saturated, data-adaptive basis under an $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$4-type complexity bound. In practical formulations, this appears either as a hard constraint $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$5 or as an equivalent penalized objective. The class is fully nonparametric in the sense used in the literature: the basis is sufficiently rich to represent nonlinearities, interactions, and discontinuities, while the complexity control comes from sectional variation rather than Euclidean dimension (Malenica et al., 2023, Munch et al., 2024).

2. Zero-order HAL and higher-order spline HAL

The original HAL construction corresponds to the zero-order case, in which basis functions are step functions. Later work generalizes the framework to higher-order smoothness classes and tensor-product spline bases. In the higher-order formulation, a $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$6-th order smoothness class $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$7 is defined by requiring recursively defined $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$8-th order Radon–Nikodym derivatives to be càdlàg with bounded variation. The corresponding $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$9-th order sectional variation norm is the sum of the variation norms of all $$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$0-th order sectional derivatives (Laan, 2023).

For zero-order HAL, one representation is

$$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$1

The discrete approximation becomes

$$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$2

For first-order HAL,

$$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$3

and for second order,

$$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$4

More generally, the higher-order theory represents the target as an infinite linear combination of tensor products of spline basis functions of order at most $$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$5, including lower-order factors on boundary faces (Zhang et al., 14 Jul 2025, Laan, 2023).

This extension is not merely cosmetic. The higher-order papers state that for first and higher order smoothness classes, pointwise asymptotic normality and uniform convergence can be established at dimension-free rate

$$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$6

up to logarithmic factors, where $$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$7. In a related inferential treatment, the $$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$8-th order spline HAL-MLE is described as smoothness-adaptive when $$\hat{\Psi}(P_n)=\arg\min_{\psi\in{\bf \Psi}}P_nL(\psi).$$9 is selected by cross-validation, while still guaranteeing a rate faster than $d$0 as long as the true function is càdlàg and has finite sectional variation norm (Laan, 2023, Laan et al., 2019).

A useful special case arises in one dimension. Recent work on univariate density estimation states that bounded sectional variation coincides with classical bounded total variation, so in $d$1,

$d$2

and for $d$3,

$d$4

The exact spline representation

$d$5

connects univariate HAL directly to classical total-variation penalized splines, local adaptive splines, and log-spline density models (Hou et al., 18 Feb 2026).

3. Statistical guarantees and convergence theory

A defining feature of HAL is that its theory is stated under minimal smoothness assumptions relative to much of nonparametric statistics. Early theory established that, in loss-based dissimilarity,

$d$6

with $d$7, and that under weak continuity conditions the estimator is uniformly consistent: $d$8 The uniform consistency result requires conditions labeled $d$9–$[0,\tau]$0, including continuity of $[0,\tau]$1, bounded loss, identifiability through zero loss-based dissimilarity, and a weak continuity condition linking pointwise convergence of functions to pointwise convergence of the loss (Laan et al., 2017).

Subsequent papers present rate claims in more specialized settings. For zero-order HAL, one paper explicitly recalls the rate

$[0,\tau]$2

and characterizes this as “almost dimension-free” because the dependence on dimension is only logarithmic in the leading rate expression (Wang et al., 11 Feb 2026). In the multi-task formulation, under square-error loss and assumptions $[0,\tau]$3 and $[0,\tau]$4,

$[0,\tau]$5

and with cross-validated $[0,\tau]$6,

$[0,\tau]$7

This is summarized there as dimension-free $[0,\tau]$8 behavior (Malenica et al., 2023).

Higher-order spline HAL sharpens this picture. The higher-order spline paper gives loss-based convergence at

$[0,\tau]$9

implying $$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$0-type convergence at rate $$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$1 up to logarithmic factors, and states pointwise asymptotic normality after normalization by $$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$2 for an oracle working model with $$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$3 (Laan, 2023). The univariate log-spline HAL density paper states three new results in $$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$4: asymptotic linearity, pointwise asymptotic normality, and uniform convergence at rate

$$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$5

up to logarithmic factors for smoothness order $$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$6 (Hou et al., 18 Feb 2026).

These results are repeatedly interpreted through the lens of plug-in efficiency. Several papers state that HAL nuisance fits converge fast enough—typically faster than $$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$7 in the relevant norm or loss—to support asymptotic linearity and semiparametric efficiency for smooth target parameters once the associated score or efficient influence curve equations are handled appropriately (Laan et al., 2021, Laan et al., 2019).

4. Computational structure, basis explosion, and principal-component reductions

The main computational limitation of HAL is the size of the basis expansion. For zero-order HAL with $$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$8 covariates and $$\psi_v = \psi(0)+ \sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{\tau_s}\left|\psi_s(du_s)\right|,$$9 observations, one paper states that the full dictionary contains

$\psi_s(u_s)=\psi(u_s,0_{-s})$0

non-intercept basis functions, while another notes that the canonical working model size is $\psi_s(u_s)=\psi(u_s,0_{-s})$1, with higher-order versions even larger (Wang et al., 11 Feb 2026, Meixide et al., 18 Mar 2026). This makes basis construction, cross-validation, and repeated lasso fitting computationally prohibitive in moderate to high dimensions.

Several papers address this bottleneck via kernel and principal-component representations. The Highly Adaptive Ridge (HAR) replaces the $\psi_s(u_s)=\psi(u_s,0_{-s})$2 constraint with an $\psi_s(u_s)=\psi(u_s,0_{-s})$3 penalty on the same dictionary, allowing kernelization through the Gram matrix

$\psi_s(u_s)=\psi(u_s,0_{-s})$4

Building on this, Principal Component based HAL (PCHAL) and related PC-HA estimators use the singular value decomposition of the HAL design matrix $\psi_s(u_s)=\psi(u_s,0_{-s})$5,

$\psi_s(u_s)=\psi(u_s,0_{-s})$6

and define low-dimensional scores

$\psi_s(u_s)=\psi(u_s,0_{-s})$7

so that fitting is performed in the orthogonal score space rather than on the original exponentially large basis (Wang et al., 11 Feb 2026).

For orthogonal scores, the resulting ridge and lasso problems admit closed forms. One paper states

$\psi_s(u_s)=\psi(u_s,0_{-s})$8

and

$\psi_s(u_s)=\psi(u_s,0_{-s})$9

for coordinates with $s$0 (Wang et al., 11 Feb 2026). Another PC-HA paper constructs an orthogonal PC basis

$s$1

from the eigenvectors of $s$2, and defines three estimators: PC-HAGL, PC-HAL, and PC-HAR, which constrain $s$3, $s$4, and $s$5, respectively (Meixide et al., 18 Mar 2026).

The principal-component line of work emphasizes that the reduction is outcome-blind and theoretically justified. The 2026 PC-HAL paper derives an excess-risk bound relative to the full $s$6-component fit,

$s$7

showing that truncation error is governed by the spectral tail of the HAL Gram operator (Wang et al., 11 Feb 2026). The PC-HA paper further states that, under complexity control, PC-HA can inherit HAL-like loss rates and score-equation properties, allowing transfer of plug-in efficiency and pointwise asymptotic normality results (Meixide et al., 18 Mar 2026).

A particularly striking observation appears in the one-dimensional ordered case, where the HAL Gram matrix has entries $s$8. The corresponding eigenvectors are discrete sine functions, yielding an explicit Fourier-type structure behind the leading principal components (Wang et al., 11 Feb 2026). This suggests that the saturated step-function dictionary of HAL possesses a much lower-rank geometry than the raw feature count $s$9 might indicate.

5. HAL in semiparametric and causal inference

HAL has been incorporated deeply into semiparametric inference, especially targeted learning. In the HAL-TMLE line, HAL-MLE provides initial nuisance estimates for outcome regression, treatment mechanism, density, or hazard components, and the targeted update is then designed to solve or approximately solve an efficient influence curve equation. One formulation of HAL-TMLE writes the target as

$$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$0

where $$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$1 is a targeted update of the initial HAL estimate $$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$2 such that

$$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$3

with $$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$4 and $$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$5 in the universal least favorable submodel case (Laan, 2017).

The asymptotic logic is standard semiparametric theory adapted to HAL rates. If the second-order remainder is $$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$6, then

$$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$7

giving asymptotic efficiency under only sectional variation assumptions on the nuisances (Laan et al., 2021, Laan, 2017). However, multiple papers stress that finite-sample inference based solely on first-order asymptotics can be anti-conservative when the second-order remainder is not negligible, motivating nonparametric bootstrap procedures for HAL-TMLE and even higher-order TMLE constructions (Laan, 2017, Cai et al., 2019, Laan et al., 2021).

HAL has also been used directly to build efficient inverse probability weighted estimators without a separate outcome model in the estimator itself. In the IPW paper, the treatment mechanism $$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$8 is estimated by HAL, and the IPW estimator

$$\psi(x)=\psi(0)+\sum_{s\subset\{1,\ldots,d\}} \int_{0_s}^{x_s} d\psi_s(u_s),$$9

is shown to be asymptotically linear and efficient when HAL is deliberately undersmoothed so that bias terms are $\psi$0 (Ertefaie et al., 2020).

A related causal-inference development is the outcome highly adaptive lasso (OHAL), which modifies the propensity-score fit by weighting the HAL penalty using outcome-regression coefficients so as to downweight instrumental basis functions. The resulting ATE estimator is a robust TMLE whose limit expansion involves

$\psi$1

and simulation evidence in that paper reports lower MSE than standard HAL-TMLE and near-nominal coverage when cross-validated standard errors are used (Ju et al., 2018).

More recent work addresses targeting after HAL selection within the finite-dimensional working model implied by the active HAL basis. “Regularized Targeted Maximum Likelihood Estimation in Highly Adaptive Lasso Implied Working Models” proposes delta-method regHAL-TMLE and projection-based regHAL-TMLE, motivated by collinearity and computational instability in relaxed HAL and full HAL-TMLE implementations. The projection-based method approximates the efficient influence curve by regularized projection onto the HAL score space and is reported to be especially stable under positivity problems and in survival-curve estimation (Li et al., 20 Jun 2025).

6. Extensions to multi-task learning, densities, hazards, and infinite-dimensional inference

HAL has been extended beyond single-task regression to settings where the target itself is structured.

The multi-task extension, MT-HAL, keeps the HAL basis expansion but shares it across tasks and applies a mixed $\psi$2 norm penalty,

$\psi$3

so that predictor-basis groups are encouraged to drop out jointly across tasks. The fitted function is constructed from indicator basis functions shared across all tasks, with optional task interactions. The empirical studies reported in that paper compare MT-HAL to MT-lasso and MT-L21 in nonlinear and linear data-generating processes. Across the main nonlinear simulations, MT-HAL is reported to achieve the lowest mean squared error in every setting, with examples such as MSEs $\psi$4, $\psi$5, and $\psi$6 versus approximately $\psi$7–$\psi$8 for MT-lasso and $\psi$9–$P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$00 for MT-L21 in $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$01, $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$02, $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$03 nonlinear cases (Malenica et al., 2023).

In survival and density estimation, HAL is used not merely as a generic regression tool but as a remedy for ill-posed empirical risk minimization over the full càdlàg class. The conditional hazard and density paper states that, for likelihood-type losses, the empirical risk minimizer over the full bounded sectional variation class may be not well-defined or inconsistent. A data-adaptive sieve version of HAL,

$P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$04

is then used to restore convexity and consistency. Under smoothness conditions, the rate

$P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$05

is established for general HAL sieve estimation, with corollaries for conditional hazard estimation under right censoring and for a new direct conditional density parametrization (Munch et al., 2024).

Plug-in estimation of non-pathwise differentiable functional targets has also become a significant application area. For the causal dose-response curve with continuous treatment, the target is

$P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$06

and HAL is used to estimate the conditional outcome regression $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$07, yielding the plug-in estimator

$P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$08

That paper emphasizes undersmoothing and smoothness-adaptive HAL fitting, and reports that the HAL-based estimator consistently outperforms GAM, polynomial regression, and npcausal in simulations, particularly for oscillatory and discontinuous dose-response curves (Shi et al., 2024).

A 2025 inferential paper extends HAL-based confidence interval construction for conditional mean functions and related infinite-dimensional parameters. It distinguishes regular HAL, relaxed HAL, and targeted HAL; introduces local and global undersmoothing criteria based on estimated bias-to-standard-error ratios; and uses delta-method Wald intervals built from the HAL working model. It also applies the same framework to conditional average treatment effect estimation through a doubly robust pseudo-outcome

$P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$09

which is then regressed on $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$10 using HAL (Zhang et al., 14 Jul 2025).

7. Interpretation, misconceptions, and methodological position

HAL is often described as a lasso, but its defining object is not a standard linear model. The lasso aspect arises because bounded sectional variation becomes an $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$11-constraint on the coefficients of an adaptive basis representation; the estimator itself is an empirical risk minimizer over a nonparametric càdlàg class. This distinction matters because the resulting guarantees are expressed in loss-based dissimilarity, sectional variation norms, and score-equation properties rather than solely in sparsity recovery or parametric restricted-eigenvalue conditions (Laan et al., 2017, Laan, 2017).

Another common misconception is that HAL is only a zero-order step-function estimator. The higher-order spline literature shows that zero-order HAL is the $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$12 or $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$13 case in a hierarchy of spline-based classes with recursively defined smoothness. In that line of work, higher-order HAL retains the same essential regularization logic but replaces step-function bases by tensor-product splines, improving approximation for smoother targets while preserving the sectional-variation interpretation of the coefficient norm (Laan et al., 2019, Laan, 2023, Zhang et al., 14 Jul 2025).

A further misconception is that cross-validated HAL is automatically suitable for inference. Several papers argue the opposite. Cross-validation selects the complexity level for predictive risk, whereas efficient inference may require undersmoothing so that bias terms or empirical score imbalances are negligible at the $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$14 scale. This issue appears in efficient IPW with HAL-estimated propensity scores, in higher-order spline HAL plug-in efficiency, and in conditional mean confidence intervals based on HAL working models (Ertefaie et al., 2020, Laan et al., 2019, Zhang et al., 14 Jul 2025).

Finally, the main criticism of HAL in practice is computational rather than statistical. The dictionary size $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$15 makes naïve implementations infeasible in moderate dimension. The recent PCHAL/PCHAR and PC-HA papers respond directly to this concern by providing outcome-blind, spectrally motivated dimension reduction with formal score-equation and excess-risk theory. This suggests that the current development of HAL is as much about computational reformulation as about new asymptotics (Wang et al., 11 Feb 2026, Meixide et al., 18 Mar 2026).

Taken together, the literature presents HAL as a general minimum-loss estimation framework over bounded sectional-variation classes, with a distinctive combination of properties: a nonparametric basis representation tied exactly to an $P_0L(\psi_0)=\min_{\psi\in{\bf \Psi}}P_0L(\psi),$16-type complexity measure; rates that are often described as near dimension-free up to logarithmic factors; compatibility with plug-in semiparametric efficiency theory; and extensibility to multi-task learning, density and hazard estimation, infinite-dimensional functionals, and modern targeted learning pipelines (Malenica et al., 2023, Munch et al., 2024, Laan et al., 2021). A plausible implication is that HAL occupies a methodological position between classical variation-penalized function estimation and modern machine-learning-based semiparametric inference: it is flexible enough to behave like a large-scale adaptive learner, yet structured enough to support exact or approximate score equations and detailed asymptotic theory.