ForestRiesz Estimator Techniques

• ForestRiesz Estimator is a family of estimators that uses the Riesz representer to enable variance reduction and √n-consistent inference in high-dimensional, semiparametric models.
• It employs random spanning forests, control variates, and stratified sampling to efficiently estimate matrix traces and inverse functionals on complex graphs.
• The estimator underpins automatic debiased machine learning by learning robust, locally constant Riesz representers via random forests in causal and semiparametric frameworks.

The ForestRiesz estimator is a family of estimators and algorithms centered on learning and exploiting the Riesz representer of a linear functional, often in high-dimensional or semiparametric settings. Its main use cases are (i) variance-reduced stochastic estimation of matrix functionals—particularly traces involving inverse SDD matrices or graph Laplacians—via random spanning forests, and (ii) automatic, robust, and debiased machine learning for statistical functionals in causal inference, semiparametric models, and sample selection, utilizing random forests to estimate Riesz representers. Across both regimes, ForestRiesz enables $\sqrt{n}$-consistent inference or near-optimal variance, leveraging properties of random forests and the Riesz representation theorem for Hilbert spaces.

1. The Riesz Representer and Linear Functionals

The Riesz representation theorem underlies ForestRiesz's statistical estimators. It states that for any continuous linear functional $\mathcal{L}(\cdot)$ on a Hilbert space of functions $g$, there exists a unique (generally unknown) function $\alpha_0$ such that for all $h$,

$\mathcal{L}(h) = \mathbb{E}[\alpha_0(Z) h(Z)].$

Many statistical and causal parameters, such as the average treatment effect (ATE) or the trace of a certain matrix function, can be expressed in this form—for appropriate regression functionals in statistical settings or as linear functionals on function spaces in numerical linear algebra (Chernozhukov et al., 2021, Bjelac et al., 13 Jan 2026, Young et al., 2024, Chernozhukov et al., 2020).

Estimators incorporating the Riesz representer often feature doubly robust correction terms, crucial for mitigating bias due to regularization, model selection, or unobserved confounding. The explicit characterization and efficient estimation of $\alpha_0$ are the foundation of ForestRiesz techniques.

2. ForestRiesz for Inverse Trace Estimation via Random Spanning Forests

The original ForestRiesz estimator in computational mathematics addresses the problem of estimating the trace

$\text{Tr}\,[K] = \text{Tr}\,[q (L + qI)^{-1}],$

where $L$ is the graph Laplacian of a weighted, symmetric, diagonally-dominant matrix, and $q>0$ serves as a regularization parameter. Forming $\mathcal{L}(\cdot)$0 and computing its trace directly is computationally prohibitive for large-scale problems. The ForestRiesz approach leverages a probabilistic representation based on random-rooted spanning forests (RSFs) (Pilavci et al., 2022):

• The distribution over forests is given by $\mathcal{L}(\cdot)$1, where $\mathcal{L}(\cdot)$2 is the number of connected components (roots), and $\mathcal{L}(\cdot)$3 are edge weights.
• The key identity is

$\mathcal{L}(\cdot)$4

• RSF samples are efficiently generated via Wilson’s loop-erased random-walk algorithm.

Variance reduction is achieved via:

1. Control variates: Incorporate a function of the random forest whose expectation is known in closed form and which correlates with $\mathcal{L}(\cdot)$5, providing substantial variance reduction.
2. Stratified sampling: Condition on the count of first-visit roots (a Poisson–binomial variable) and partition into strata, sampling forests conditioned on each stratum. This isolates the primary source of variance, particularly for graphs with heterogeneous degree distribution.

These modifications enable computational schemes that, at fixed error, are as efficient as or superior to state-of-the-art alternatives, especially on large or irregular graphs (Pilavci et al., 2022).

3. ForestRiesz in Debiased Machine Learning and Causal Inference

ForestRiesz has been generalized for use in automatic debiased machine learning, especially for estimating linear functionals of high-dimensional regression objects in causal inference and semiparametric statistics. Here, the ForestRiesz algorithm learns the Riesz representer $\mathcal{L}(\cdot)$6 via random forests:

• For samples $\mathcal{L}(\cdot)$7, where $\mathcal{L}(\cdot)$8 are treatment variables and $\mathcal{L}(\cdot)$9 are covariates, and $g$0,
• The target is $g$1, with $g$2, $g$3 linear in $g$4.
• The doubly robust estimator is

$g$5

• ForestRiesz posits $g$6, where $g$7 is locally constant within random-forest leaves, and fits $g$8 via local normal equations minimizing the empirical Riesz loss (Chernozhukov et al., 2021, Bjelac et al., 13 Jan 2026, Young et al., 2024).

Robustness and asymptotic normality are guaranteed if the product of $g$9 errors $\alpha_0$0. This is achieved by coupling honest random forest learners for $\alpha_0$1 and $\alpha_0$2 and leveraging cross-fitting.

4. Algorithmic and Implementation Details

Four core ForestRiesz algorithms have been developed, tailored to their application domain:

• Random Spanning Forest Estimator (Numerical Linear Algebra): Draws samples via Wilson's algorithm; uses control variates or stratified sampling for variance reduction; per-sample time is $\alpha_0$3, where $\alpha_0$4 is the number of edges (Pilavci et al., 2022).
• Locally Linear Riesz Forests (Statistical Estimation, Causal Inference): Grows forests splitting on covariates, fits a piecewise-constant coefficient vector in each leaf, and aggregates over trees; employs low-dimensional features in treatment for stability; parallelizable and scales as $\alpha_0$5, with $\alpha_0$6 trees and feature dimension $\alpha_0$7 (Chernozhukov et al., 2021, Young et al., 2024).
• Adversarial ForestRiesz: Formulates Riesz estimation as a regularized two-player min-max zero-sum game between linear functionals and candidate representers, iteratively updating random-forest parameterizations (Chernozhukov et al., 2020).

Implementation is robust to high-dimensional features, model misspecification, and challenging sample selection settings. Practical hyperparameters include large forests (500–2,000 trees in standard settings), modest tree depth, and minimum leaf size calibrated by cross-validation or validation loss minimization.

5. Theoretical Properties and Guarantees

ForestRiesz estimators are supported by advanced nonasymptotic and asymptotic theory:

• Consistency and asymptotic normality are ensured under conditions mirroring those for honest random forests and cross-fitted estimators: specifically, estimation error for the representer and regression function must be $\alpha_0$8 in $\alpha_0$9, ensuring the product is second-order negligible (Chernozhukov et al., 2021, Young et al., 2024).
• Variance reduction for inverse trace estimation is quantifiable: 2–5× for control variates on regular graphs, 5–10× for stratified sampling on power-law graphs (Pilavci et al., 2022).
• In causal learning, resulting estimators are semiparametrically efficient or “robust efficient” (in the ROSE sense) uniformly over large function classes—achieving the minimal variance among all robust influence functions of the target moment (Young et al., 2024).

6. Practical Impact and Applications

ForestRiesz estimators have been deployed in:

• Large-scale numerical linear algebra: Optimization of MCMC and kernel methods requiring trace estimation of resolvents. ForestRiesz matches or beats state-of-the-art iterative solvers under large-scale and irregular graph settings (Pilavci et al., 2022).
• Causal inference with non-random sample selection: Accurate estimation of treatment effects under non-random outcome observability, yielding larger and more robust estimates of the gender wage gap, for example, when compared to standard double machine learning with inverse probability weighting (Bjelac et al., 13 Jan 2026).
• General semiparametric regression: The ROSE random forest procedure delivers substantial variance reduction when standard efficient estimators are unstable or infeasible due to poor nuisance estimation. Applications include partially linear models, heteroscedastic designs, and high-dimensional policy effect estimation (Young et al., 2024, Chernozhukov et al., 2021).
• Automatic debiasing frameworks: ForestRiesz provides a “plug-and-play,” black-box route to doubly robust and orthogonally scored estimation, adaptable to arbitrary linear functionals (Chernozhukov et al., 2021).

7. Related Estimators, Limitations, and Extensions

ForestRiesz methods are closely related to:

• Hutchinson’s stochastic trace estimators in matrix functionals, but with improved variance via combinatorial graph structure.
• Doubly robust/debiased machine learning frameworks (DML, targeted ML) but distinguish themselves by directly learning the Riesz representer via forests or adversarial algorithms, eliminating the need for analytical derivation of orthogonal scores.
• ROSE (Robust Semiparametric Efficient) random forests as a generalization to efficient influence function estimation (Young et al., 2024).
• Adversarial methods that generalize function classes for the representer beyond trees, extending to neural networks or RKHS (Chernozhukov et al., 2020).

Limitations include requirement of sufficient overlap and rich feature support for honest splitting, and sensitivity to the chosen feature map for nonparametric functionals. Nonetheless, empirical and theoretical analyses demonstrate stability and robustness superior to direct propensity score inversion or classical estimators, especially in finite samples and highly nonlinear or high-dimensional contexts (Bjelac et al., 13 Jan 2026, Chernozhukov et al., 2021, Young et al., 2024, Pilavci et al., 2022).

---

Key Citations:

• Variance Reduction for Inverse Trace Estimation via Random Spanning Forests (Pilavci et al., 2022)
• RieszNet and ForestRiesz: Automatic Debiased Machine Learning with Neural Nets and Random Forests (Chernozhukov et al., 2021)
• Automatic debiased machine learning and sensitivity analysis for sample selection models (Bjelac et al., 13 Jan 2026)
• ROSE Random Forests for Robust Semiparametric Efficient Estimation (Young et al., 2024)
• Adversarial Estimation of Riesz Representers (Chernozhukov et al., 2020)