RieszBoost: Gradient Boosting for Riesz Regression (2501.04871v2)

Abstract: Answering causal questions often involves estimating linear functionals of conditional expectations, such as the average treatment effect or the effect of a longitudinal modified treatment policy. By the Riesz representation theorem, these functionals can be expressed as the expected product of the conditional expectation of the outcome and the Riesz representer, a key component in doubly robust estimation methods. Traditionally, the Riesz representer is estimated indirectly by deriving its explicit analytical form, estimating its components, and substituting these estimates into the known form (e.g., the inverse propensity score). However, deriving or estimating the analytical form can be challenging, and substitution methods are often sensitive to practical positivity violations, leading to higher variance and wider confidence intervals. In this paper, we propose a novel gradient boosting algorithm to directly estimate the Riesz representer without requiring its explicit analytical form. This method is particularly suited for tabular data, offering a flexible, nonparametric, and computationally efficient alternative to existing methods for Riesz regression. Through simulation studies, we demonstrate that our algorithm performs on par with or better than indirect estimation techniques across a range of functionals, providing a user-friendly and robust solution for estimating causal quantities.

• The paper presents a novel gradient boosting method that directly estimates Riesz representers, bypassing the need for explicit analytical forms.
• Simulation studies demonstrate that RieszBoost outperforms indirect methods with lower RMSE and MAE in estimating ATE and ATT.
• The approach simplifies implementation on tabular data and offers promising extensions for high-dimensional causal inference applications.

An Overview of RieszBoost: Gradient Boosting for Riesz Regression

The paper introduces "RieszBoost," a methodological advancement for estimating Riesz representers through gradient boosting, addressing notable challenges in causal inference. The primary focus is on estimating linear functionals of conditional expectations pertinent to causal queries, such as average treatment effects (ATE) and longitudinal modified treatment policies. Traditionally, the Riesz representation theorem facilitates expressing these functionals, and the Riesz representer plays a critical role in doubly robust estimation. However, the conventional approach to estimate the representer involves deriving its explicit analytical form and is sensitive to practical positivity issues, potentially inflating variance and confidence intervals. RieszBoost circumvents these hurdles, offering an innovative and pragmatic alternative without requiring explicit analytical forms.

Methodological Contributions

1. Direct Estimation via Gradient Boosting: RieszBoost employs a novel gradient boosting strategy, moving beyond traditional estimations that rely on deriving the Riesz representer's analytical structure. By directly modeling the Riesz representer, the method streamlines the estimation process, particularly benefiting scenarios with complex causal estimands or tabular data sets.
2. Simulation Studies and Comparative Performance: The authors validate RieszBoost through rigorous simulation studies. It consistently matches or surpasses indirect estimation methods concerning accuracy and robustness over diverse functionals, notably in estimating the ATE and the Average Treatment Effect among the Treated (ATT). The simulations underscore its efficacy, showcasing lower root mean squared error (RMSE) and mean absolute error (MAE) in Riesz representer estimation compared to indirect methods which estimate nuisance parameters.
3. Implementation Considerations: RieszBoost is versatile, particularly with tabular datasets. It efficiently leverages gradient boosting—a machine learning technique renowned for its competitiveness in performance and ease of training over neural networks for similar data contexts. The method demands minimal tuning, primarily concerning the number of boosting iterations, learning rates, and tree depth.
4. Augmented Approach: RieszBoost introduces an innovative data augmentation tactic. It addresses the distinctive requirement to evaluate the Riesz representer at counterfactual data points, enhancing algorithm feasibility and reliability without drastically altering typical boosting frameworks.

Practical and Theoretical Implications

The introduction of RieszBoost holds substantial theoretical and practical implications for causal inference. Theoretically, it alleviates the necessity for explicit derivations of the Riesz representer, aligning with the more general balancing weights literature. Practically, by simplifying the estimation of complex causal parameters, RieszBoost can be employed across a broad spectrum of empirical studies where causal inference is pivotal, leveraging its computational efficiency and ease of implementation.

Future Directions

The paper paves the way for potential advancements in Riesz regression methodologies, especially within high-dimensional and complex modeling scenarios where traditional approaches face computational or conceptual challenges. There's a scope for extending these approaches in different inference settings, such as stratified or conditional causal effects. Additionally, integrating RieszBoost into other machine learning paradigms could further solidify its applicability and adoption, potentially exploring its synergy with reinforcement learning and dynamic treatment regimes.

In conclusion, RieszBoost represents a significant step forward in the nonparametric and computationally efficient estimation of causal quantities, expanding the toolkit available to researchers in biostatistics and beyond. It exemplifies an advancement in methodological approaches to address nuanced challenges in causal inference, promoting more accessible and reliable analytic procedures.