How Many Trees in a Random Forest? A Revisited Approach with Plateau Search and Optuna Integration

Abstract: Hyperparameter optimization (HPO) for Random Forest faces a specific difficulty in tuning the number of trees: the predictive score typically improves monotonically with ensemble size, so standard methods such as Tree-structured Parzen Estimator (TPE) and Hyperband require a predefined search range and often drive the estimate toward its right boundary. Early-stopping strategies avoid fixing such a range, but can be sensitive to score noise and prone to premature stopping. To address this, we propose an integrated triplet-based plateau-search algorithm that removes the number of trees from the direct TPE search space and still exploits information accumulated across HPO trials. The method adaptively tracks a near-minimal sufficient ensemble size by monitoring relative changes in the out-of-bag (OOB) score across a triplet of forest sizes and shifting this triplet accordingly. This yields an automated and user-interpretable procedure based on a tolerance parameter. We also provide a theoretical analysis: we relate the proposed relative OOB-score criterion to the gap between the current and limiting scores, and derive an asymptotic variance estimate for the corresponding OOB-based absolute relative difference. Experiments show that the selected number of trees can differ substantially from the common heuristic: for most classical benchmark datasets it is smaller, whereas for some high-dimensional bioinformatics datasets, such as Arcene and Dorothea, it is larger. The source code and reproducible experiments are available at https://github.com/lange-am/rf_plateau_hpo.

• The paper introduces a triplet-based method that uses plateau detection on out‐of‐bag scores to determine the optimal number of trees.
• It integrates the Optuna TPE framework to jointly tune tree count and RF hyperparameters, linking finite performance to an infinite tree limit.
• Empirical results across diverse datasets demonstrate that the approach improves computational efficiency and model stability, especially in high-dimensional settings.

Triplet-Based Adaptive Plateau Search for Random Forests: A Detailed Technical Analysis

Introduction and Motivation

Optimizing the number of trees ($T$) in Random Forests (RF) remains a critical but underexplored aspect of efficient model construction, especially as practitioners often default to heuristic choices or upper-bounded hyperparameter sweeps. The number of trees not only impacts predictive performance stability but is essential for consistent variable importance measures—especially in high-dimensional or correlated settings. The paper "How Many Trees in a Random Forest? A Revisited Approach with Plateau Search and Optuna Integration" (2606.03549) introduces a triplet-based adaptive algorithm that integrates plateau detection for $T$ into the Optuna TPE hyperparameter optimization (HPO) framework, moving away from traditional fixed-range searches.

Key contributions of the work include:

• Defining a plateau-based sufficiency criterion for $T$ using relative changes in the out-of-bag (OOB) score across a multiplicatively spaced triplet of tree counts.
• Joint optimization of $T$ and subordinate RF hyperparameters (e.g., max depth, max features) without a priori upper and lower tree limits.
• Theoretical analysis linking the plateau criterion to the gap between finite $T$ and limiting (infinite tree) performance, with variance analysis on the OOB-based stopping rule.
• Empirical validation across a suite of tabular and bioinformatics datasets, demonstrating that optimal $T$ can be both lower (classical tabular data) and substantially larger (high-dimensional biology) than standard heuristics.
• Open-source Python implementation and reproducibility focus.

Motivation and Limitations of Existing Approaches

Standard HPO frameworks (e.g., TPE, Hyperband) require a fixed $[T_{\min}, T_{\max}]$ search interval for the number of estimators. As the RF score typically improves monotonically with $T$, such methods often bias toward $T_{\max}$, providing no guarantee of sufficiency or efficiency and risking computational overprovision. Early stopping and doubling rules are sensitive to OOB noise and often bias $T$ downward due to error-prone monotonic traversal. Critically, these methods lack rigorous theoretical linkage between the sufficiency threshold and true limiting behavior.

Moreover, existing heuristics decouple $T$0 selection from other tree hyperparameters (tree depth, mtry, splitter, etc.), disregarding interdependencies. Empirical studies and theoretical works have shown that these parameters interact in complex ways, particularly in high-dimensional or correlated-feature regimes.

The Triplet-Based Plateau Search Algorithm

The core contribution is an adaptive search for $T$1 based on monitoring the relative OOB-score change over three tree counts: $T$2, $T$3, and $T$4, where $T$5 is a multiplicative scale factor. At each HPO trial, RFs are trained at these three ensemble sizes, with corresponding OOB scores $T$6, $T$7, $T$8. The algorithm evaluates the plateau via two metrics:

$T$9

Scenarios:

1. B insufficient: Both $T$0 and $T$1 exceed the user-chosen tolerance $T$2, so the triplet shifts right.
2. B sufficient: $T$3, $T$4 — plateau is just attained.
3. B excessive: Both quantities $T$5 — triplet shifts left, indicating possible overprovision.
4. Pathological random fluctuation: Handled to mitigate underestimation bias.

A revisit phase further shifts left for the best trial to ensure minimal sufficiency.

(Figure 1)

Figure 1: Three plateau search scenarios shown as triplet points $T$6 on the RF OOB-score curve: right shift (insufficient), stay (plateau reached), left shift (excessive), demarcated by vertical $T$7 and $T$8 gaps.

This criterion is robust to OOB noise—as it leverages the inherent randomness of the OOB mechanism and repetitions of HPO trials—thus avoiding redundant nested resampling.

A practical consequence is that the user no longer needs to set arbitrary $T$9 values. Instead, tolerance $T$0 provides a transparent, domain-relevant knob that can be aligned with the OOB metric’s empirical quantization.

Theoretical Analysis and Plateau Criterion Interpretation

The paper derives a power-law asymptotic expansion for the OOB score with trees:

$T$1

The plateau detection's stopping criterion can be related to the finite-vs-infinite tree performance gap:

$T$2

Thus, the tolerance parameter $T$3 (adjusted for $T$4 and $T$5) directly bounds the remaining discrepancy to the infinite-ensemble limit. For accuracy and ROC-AUC, discrete quantization of the OOB metric guides practical lower limits on $T$6. The variance of the plateau statistic decays as $T$7, establishing both theoretical and empirical justification for stochastic but consistent convergence.

Experimental Results

A comprehensive benchmark was conducted across 12 datasets (tabular, financial, and high-dimensional biological), including credit scoring and microarray datasets with hundreds to over 100,000 features. Key findings include:

• On most classical datasets, the plateau method leads to smaller $T$8 than standard range-limited HPO, saving substantial computation.
• On "hard" datasets (e.g., Arcene, Dorothea), the procedure reliably identifies the need for much larger forests, sometimes exceeding standard $T$9 values by orders of magnitude.
• Joint tuning of $T$0 and tree-structural hyperparameters consistently outperforms decoupled or sequential tuning strategies.
• Sensitivity analysis with respect to $T$1 and $T$2 shows expected trade-offs: smaller tolerances lead to larger forests and higher stability at increased cost, while larger $T$3 steps speed up traversal but risk coarser resolution.
• Compared to Hyperband and Early-Stopping baselines, PLATEAU reduces both the run-to-run variance and (in most regimes) wall-clock time when targeting an empirically reasonable error floor.

  Figure 2: Trajectories of central triplet point $T$4 across HPO trials for different datasets, showing adaptation speed and stability; background colormap reflects empirical frequency of selected tree counts.

  

  Figure 3: Sensitivity of best OOB performance to tolerance parameter $T$5; smaller $T$6 yields higher accuracy/lower RMSE but at increased computational expense.

  

  Figure 4: Comparative runtime and number of trees selected for TPE, Hyperband, Early-Stop, and PLATEAU methods; PLATEAU often results in shorter run times and parsimonious forests, except in "hard" high-dimensional cases with true demand for large ensembles.

Statistical evaluations (Table columns) confirm that increased trial budgets benefit both baseline and novel approaches. Joint tuning is critical, especially in high-dimension/low-sample contexts.

Implications and Future Directions

This work substantially advances Random Forest model construction by providing a theoretically justified, empirically robust method for selecting the number of trees required for sufficient predictive and interpretative stability. The plateau-based stopping rule removes the need for arbitrary budget caps while exploiting OOB statistics and the stochastic exploration inherent to Bayesian HPO. The implications are amplified in bioinformatics and other high-dimensional science domains, where reproducibility of feature importances is paramount and standard heuristics are inadequate.

The scope for further development includes:

• Incorporating stability criteria directly for variable importance convergence and extending the joint optimization to post-hoc forest growth personalized for interpretability demands.
• Extension to distributed or parallel RF implementations where computational cost scales superlinearly with $T$7.
• Rigorous large-$T$8, low-$T$9 benchmarking across genomics, proteomics, and medical datasets, where variable importance is as critical as classification performance.

Conclusion

The triplet-based adaptive plateau search provides a rigorous, interpretable, and computationally efficient solution to previously arbitrary practices in Random Forest ensemble size selection. It promotes stability and reproducibility in model predictions and variable importance, especially for non-classical data regimes. The methodology will likely become a foundational approach for practitioners and researchers requiring high-confidence, resource-efficient RF modeling in modern data analysis pipelines.

(2606.03549)