CART Random Forests as Sequential Allocation over Random Opportunity Sets: A Stochastic-Control Theory of Ensemble Risk

Abstract: CART random forests are among the most widely used modern predictive methods, with well-documented empirical success. Yet, at the mechanistic level, the algorithm is often treated as a black box because of its complexity. In this paper, we develop a stochastic-control perspective on feature-subsampled CART random forests, named CART random opportunity-set allocation (CART-ROSA). At each node, the random subset of features is interpreted as a random feasible action set, and the CART split rule as a masked-action allocation policy. This policy induces a controlled stochastic process over informative split-count states, whose terminal law determines both single-tree error and cross-tree interaction terms in the forest mean squared error (MSE). Such representation opens the black box of CART-forests by separating two design levers: the informative-opportunity rate induced by feature subsampling, and the contraction strength from the within-mask split policy. We establish that the CART policy is locally stabilizing: it contracts imbalances in informative split allocations and concentrates terminal tree geometry. At the system level, however, it can be globally suboptimal for the forest objective. Specializing to the linear model, we derive the MSE risk expansion explicitly. Our results show how an operations-research perspective makes tractable a theoretical gap difficult to access from the standard algorithmic description of CART forests.

• The paper presents a novel stochastic-control framework that models CART random forests as sequential allocations over random opportunity sets.
• It rigorously analyzes local stabilization through masked-action policies, demonstrating exponential compression of split allocation imbalances.
• The study reveals that while greedy split rules stabilize locally, they are globally suboptimal, highlighting a crucial bias–variance tradeoff.

Stochastic Control Framework for CART Random Forests Ensemble Risk

Introduction

The paper "CART Random Forests as Sequential Allocation over Random Opportunity Sets: A Stochastic-Control Theory of Ensemble Risk" (2605.26675) develops a stochastic-control-based theoretical framework for analyzing feature-subsampled CART random forests (RFs). The main conceptual advance is to recast the recursive tree-building process—as used in population-level CART implementations—into a sequential allocation over random opportunity sets, labeled as CART-ROSA. This formalism unveils two primary design levers in random forest construction: (1) the informative opportunity rate determined by feature subsampling, and (2) the contraction strength induced by the within-mask split policy. The paper rigorously characterizes local stabilization properties of population CART and proves that such local logic does not necessarily achieve global optimality in ensemble-level mean squared error (MSE).

Masked-Action Stochastic-Control Formulation

The key technical insight is the modeling of the CART random forest construction as a finite-horizon stochastic control problem:

• State variable: The split-count vector records the allocation of splits among the features, encoding the geometry of the branch.
• Action mask: At each node, a random feasible subset of features is sampled via feature subsampling; this forms the random opportunity set $U_t$.
• Policy: The standard CART greedy split rule (maximizing impurity decrease) acts as a masked-action allocation policy on the current feasible set.
• Transition: The split-count state is deterministically updated according to the selected feature.
• Terminal law: The distribution of terminal split-count vectors (over depth-$\ell$ branches) directly determines both single-tree prediction error and cross-tree interaction terms in forest-level MSE.

This representation allows separation of "opportunity" ($q$, the probability an informative coordinate is available for splitting at a node) from "exploitation" ($\kappa$, the contraction strength or stabilizing force in split allocation policy).

Local Stabilization—Contractive Dynamics of Population CART

Within a sparse linear midpoint model (i.i.d. uniform covariates, sparse informative features), the population-level CART split rule is shown to act as a balancing policy. When an informative opportunity is exposed, the greedy policy prefers coordinates with lower accumulated split counts (modulo deterministic signal shift). This yields:

• First-order equalization: The split counts across informative features converge to the same ratio, matching the informative opportunity rate $q$ divided equally among informative features.
• Second-order exponential stabilization: The imbalance in split allocation (measured by the Euclidean norm of deviation from perfect balance) is exponentially compressed, and the process remains tightly concentrated near its equilibrium.

The negative drift induced by the greedy policy on the imbalance is quantified, and exponential moment bounds are established.

Local–Global Nonalignment: Suboptimality of Greedy for Ensemble MSE

A fundamental limitation is exposed: the local one-step CART split rule, though stabilizing, does not optimize the global (ensemble-level) MSE objective. The ensemble MSE contains both single-tree bias terms and cross-tree interaction (overlap) terms. Through an explicit Bellman-type marginal-cost certificate, the authors construct a counterexample wherein locally greedy allocation is strictly suboptimal for the terminal-law ensemble objective. Therefore, greedy CART is not guaranteed to be a local minimizer of the forest-level MSE—a formal violation of Bellman optimality is proven.

Policy Space and Exploration–Exploitation Tradeoff

The framework allows systematic exploration of policy families:

• Exploratory benchmark policy: Uniform random allocation among exposed informative coordinates yields the same first-order equilibrium, but much weaker stabilization (diffusive multinomial regime).
• Mixture policies: Interpolation via an $\alpha$-mixture between greedy and exploratory policies tunes the contraction coefficient $\kappa$ and allows fine-grained control of stabilization. Theoretical analysis shows bias–variance tradeoffs: stronger stabilization (higher $\kappa$) improves bias but may reduce cross-tree diversity, detrimental for variance.

Statistical Mapping: Terminal Laws to MSE Functionals

The paper rigorously translates terminal split-count distributions into explicit non-asymptotic MSE bounds for both single-tree and ensemble RFs, under the sparse linear midpoint model. Three key MSE functionals are identified:

• Single-tree bias: Expectation over $2^{-2N_{\ell,j}}$ (exponential in split counts on informative coordinates).
• Cross-tree bias: Expectation over $2^{-2\max\{N_{\ell,j},N'_{\ell,j}\}}$.
• Cross-tree variance/overlap: Expectation over $\ell$0.

Equilibrium replacement utilizes multinomial proxies and Poisson-kernel representations for tractable analysis. Strong exponential stabilization enables effective dimension reduction (collapse of variance proxy), illustrating how stabilization modifies the effective geometry of terminal cells in the ensemble.

Practical Regime Map and Implications

The two-lever design space ($\ell$1, $\ell$2) is mapped to practical guidance:

• Bias-dominated regimes: Favor strong stabilization/high $\ell$3 to accelerate bias decay.
• Variance-dominated regimes: Soften exploitation (lower $\ell$4) to maximize diversity and minimize variance.
• Sparse/high-dimensional settings: Prioritize increasing $\ell$5; excessive exploitation cannot compensate for rare informative exposure.
• Mixed regimes: Jointly tune both levers via mixture policies.

Empirical score-window policies are introduced for practical implementation, and simulation results validate the theoretical guidance. When feature subsampling is sufficient, exploitation intensity can be optimized to balance bias and variance depending on signal regime.

Figure 1: Empirical MSE heatmaps for honest forests under low and high-SNR, showing the interplay of feature subsampling and split-policy softness on prediction performance.

Conclusion

This paper establishes a rigorous stochastic-control theoretic foundation for mechanism-level analysis of feature-subsampled CART random forests. The masked-action sequential allocation framework facilitates separation of opportunity generation and within-mask split policy. Population CART is shown to be locally stabilizing, leading to exponential compression of split allocation imbalance, but is not globally optimal for ensemble MSE. The terminal-law perspective enables explicit bias–variance characterization and clarifies the need for policy-level tuning in practical RF design. The theoretical and empirical results collectively suggest that ensemble performance critically depends on a two-lever tuning—opportunity rate $\ell$6 and contraction strength $\ell$7—with local stabilization and cross-tree diversity as competing objectives. The methodology opens avenues for further theoretical exploration, especially beyond linear midpoint models and toward general covariate distributions and empirical implementations.