Triplet-Based Plateau-Search Algorithm

• The paper introduces a triplet-driven adaptive procedure that replaces direct hyperparameter search over ensemble size with geometric plateau tests on OOB scores.
• It utilizes a geometric triplet of forest sizes and adaptive shift rules (original and modified) to monitor score changes and prune non-promising trials.
• Stationary-distribution analysis models the central ensemble size as a stochastic process, guiding practical convergence and efficiency in various high-dimensional scenarios.

Searching arXiv for the cited triplet-based plateau-search and search-by-triplet papers to ground the article in the relevant literature. The integrated triplet-based plateau-search algorithm is a triplet-driven adaptive procedure for selecting Random Forest ensemble size inside hyperparameter optimization by monitoring out-of-bag (OOB) score changes across a geometric triplet of forest sizes and shifting that triplet until a plateau is reached within a tolerance parameter. In the primary formulation, the number of trees is removed from the direct Optuna/TPE search space and is instead updated internally from trial to trial through local plateau tests on $L=B/sf$, $B$, and $R=B\cdot sf$, with $sf>1$ a fixed scale factor (Porvatov et al., 2 Jun 2026). A later stationary-distribution analysis formalizes the central ensemble size as a stochastic process on a geometric grid rather than a deterministic stopping point, deriving equilibrium location and spread under a folded-normal approximation for OOB score differences (Dukhovny et al., 29 Jun 2026). A separate line of work on local particle tracking at LHCb also uses triplet-based search, but explicitly notes that the published "Search by triplet" algorithm does not include an explicit plateau-search mechanism; in that setting, plateau awareness is presented only as a proposed integration (Pérez et al., 2022).

1. Problem setting and defining features

Random Forest tuning has a specific difficulty in selecting the number of trees because the predictive score typically improves monotonically with ensemble size, so classical HPO procedures such as Tree-structured Parzen Estimator and Hyperband require a predefined search interval $[T_{\min},T_{\max}]$ and often drive the estimate toward its right boundary (Porvatov et al., 2 Jun 2026). Early-stopping strategies avoid fixing such a range, but can be sensitive to score noise and prone to premature stopping. The integrated triplet-based plateau-search algorithm addresses this by replacing direct search over n_estimators with a geometric triplet test on OOB scores and by updating the triplet adaptively across HPO trials (Porvatov et al., 2 Jun 2026).

The method is “integrated” in a precise sense. n_estimators is not sampled by TPE; instead, TPE samples the remaining Random Forest hyperparameters, while each trial trains three nested forests at $L$, $B$, and $R$, computes their OOB scores, and applies a plateau test. Trials where no plateau is reached are pruned through plateau-specific pruning, and only trials that satisfy the right-plateau condition contribute an objective value (Porvatov et al., 2 Jun 2026). This design uses information accumulated across trials while removing the ensemble size from the explicit Bayesian search space.

The operational criterion is local and relative. With OOB scores $S_L$, $S_B$, and $B$0, the algorithm computes

$B$1

A tolerance $B$2 defines whether the score change is still material or already within the plateau regime (Porvatov et al., 2 Jun 2026). This gives a user-interpretable rule: $B$3 is treated as sufficient when the right-side relative change is no larger than $B$4, and excessive when both sides satisfy the same bound.

A central conceptual point is that the algorithm is not merely a stopping heuristic. The stationary-distribution formulation shows that after the remaining hyperparameters have stabilized, the central triplet point need not converge to a deterministic value; instead, it fluctuates around a stationary regime determined by $B$5 and the scale factor (Dukhovny et al., 29 Jun 2026). This directly counters the common misconception that a plateau test must settle at a single terminal ensemble size.

2. Triplet mechanics and update rules

The triplet is geometric rather than additive. For a current central ensemble size $B$6, the candidate sizes are $B$7 in the stationary-distribution formulation and $B$8 in the Optuna/TPE formulation, with $B$9 or $R=B\cdot sf$0 fixed (Dukhovny et al., 29 Jun 2026, Porvatov et al., 2 Jun 2026). The geometric spacing preserves a constant relative separation at large $R=B\cdot sf$1, whereas fixed additive steps would become indistinguishable as the ensemble grows.

The decision logic is based on whether the left and right differences exceed the tolerance. The primary Optuna/TPE formulation uses three principal cases plus a noise-induced fourth case. The stationary-distribution paper gives both the original rule and a symmetric modified variant for the same triplet test (Dukhovny et al., 29 Jun 2026).

Plateau-test outcome	Original rule	Symmetric modified rule
$R=B\cdot sf$2	shift right	shift right
$R=B\cdot sf$3	stay	stay
$R=B\cdot sf$4	shift left	shift left
$R=B\cdot sf$5	shift right	stay

With $R=B\cdot sf$6, the update is

$R=B\cdot sf$7

In the original rule, the mixed case is mapped to $R=B\cdot sf$8, which the paper characterizes as conservative; in the symmetric modified rule, the same case is mapped to $R=B\cdot sf$9 (Dukhovny et al., 29 Jun 2026). In the Optuna/TPE implementation, the asymmetric case $sf>1$0, $sf>1$1 is explicitly treated as shift right to counteract early stopping bias (Porvatov et al., 2 Jun 2026).

To avoid cumulative rounding, the implementation specifies explicit shift rules. Right shift updates $sf>1$2; left shift updates $sf>1$3. In “stay” and “excessive” cases, the trial returns $sf>1$4, favoring the smallest sufficient $sf>1$5 because $sf>1$6 guarantees $sf>1$7 within tolerance (Porvatov et al., 2 Jun 2026).

This update structure makes the algorithm distinct from monotone sweep-based early stopping. Rather than scanning only to the right and halting at the first acceptable plateau, it allows right shifts, stays, and left shifts. A plausible implication is that the procedure can correct local underestimation and overestimation errors induced by OOB noise, which is exactly the phenomenon later formalized through the stationary regime (Dukhovny et al., 29 Jun 2026).

3. Stationary-distribution theory

The stationary-distribution analysis models the grid index $sf>1$8 through

$sf>1$9

and treats the evolving center $[T_{\min},T_{\max}]$0 as a birth-death Markov chain on this geometric grid (Dukhovny et al., 29 Jun 2026). At level $[T_{\min},T_{\max}]$1, the left- and right-gap plateau-pass probabilities are

$[T_{\min},T_{\max}]$2

Under a factorized approximation for the two tests at the same level, the original rule uses

$[T_{\min},T_{\max}]$3

while the modified rule uses

$[T_{\min},T_{\max}]$4

$[T_{\min},T_{\max}]$5

The stationary distribution $[T_{\min},T_{\max}]$6 satisfies local balance

$[T_{\min},T_{\max}]$7

with ratio form

$[T_{\min},T_{\max}]$8

and product-form masses

$[T_{\min},T_{\max}]$9

Under the paper’s variance asymptotics, $L$0 as $L$1, implying $L$2, $L$3, and therefore existence and uniqueness of a normalizable stationary distribution for any fixed $L$4 (Dukhovny et al., 29 Jun 2026).

The analytical approximation is built on a tail model for the conditional mean score,

$L$5

together with nested warm-start covariance scaling

$L$6

and an approximately Gaussian signed relative gap whose mean decays faster than its standard deviation (Dukhovny et al., 29 Jun 2026). Under the leading centered folded-normal approximation,

$L$7

Because the left gap involves the smaller forest $L$8, it is noisier, with

$L$9

(Dukhovny et al., 29 Jun 2026).

Equilibrium follows from the zero-drift condition $B$0 at the stationary center. The paper derives

$B$1

for the original rule, and

$B$2

for the modified rule. Substituting the variance asymptotics yields

$B$3

and therefore

$B$4

for both rules (Dukhovny et al., 29 Jun 2026).

The same analysis quantifies spread. On the grid scale,

$B$5

or, equivalently from discrete local balance,

$B$6

After conversion to ensemble-size scale by the delta method,

$B$7

so that

$B$8

The leading relative spread

$B$9

is independent of $R$0 and controlled by $R$1 and the update rule (Dukhovny et al., 29 Jun 2026). This is the key theoretical basis for interpreting plateau search as a stationary stochastic process rather than a deterministic stopping rule.

4. Relation to the infinite-forest limit and OOB-noise scaling

The Optuna/TPE paper relates the local triplet criterion to the gap between the current OOB score and the infinite-forest limit (Porvatov et al., 2 Jun 2026). Under

$R$2

and with $R$3, it proves

$R$4

and therefore, if

$R$5

then

$R$6

The observed local plateau gap thus controls the relative distance to the limiting score up to the multiplicative factor $R$7 (Porvatov et al., 2 Jun 2026).

The same paper derives an asymptotic variance estimate for the signed relative difference and for the absolute relative difference used by the algorithm. Assuming $R$8 is approximately bivariate normal with means $R$9, variances $S_L$0, covariance $S_L$1, and $S_L$2,

$S_L$3

With

$S_L$4

and $S_L$5, the asymptotic variance becomes

$S_L$6

For the absolute relative difference actually used in the triplet test, and under the tail model with $S_L$7,

$S_L$8

(Porvatov et al., 2 Jun 2026).

These formulas give the algorithm a direct statistical interpretation. OOB variability decays like $S_L$9 in standard deviation, while the stationary center scales like $S_B$0. This suggests that smaller tolerances do not simply refine a deterministic solution; they move the stationary regime toward larger ensembles and retain a non-vanishing relative spread on the geometric grid (Dukhovny et al., 29 Jun 2026, Porvatov et al., 2 Jun 2026).

5. Integration with Optuna/TPE and empirical behavior

The practical implementation removes n_estimators from the Optuna search space and lets TPE sample only the remaining Random Forest hyperparameters, such as max_features, max_depth, min_samples_leaf, min_samples_split, and optionally the split criterion (Porvatov et al., 2 Jun 2026). Each trial trains forests at $S_B$1, $S_B$2, and $S_B$3 using warm_start=True, so the total number of trees built per trial is $S_B$4, giving time complexity $S_B$5 rather than $S_B$6 (Porvatov et al., 2 Jun 2026).

The integrated implementation exposes three APIs: tune_rf_oob() as a classic TPE baseline with n_estimators_range, tune_rf_oob_bohb() as a Hyperband-like baseline, and tune_rf_oob_plateau() as the integrated triplet-based method (Porvatov et al., 2 Jun 2026). The key parameters of tune_rf_oob_plateau() are n_estimators_start = T0, scale_factor = sf, delta = \varepsilon, and max_trees; the return values are a fitted Random Forest, an Optuna study, best_n_estimators, and a flag indicating whether any trial reached a plateau (Porvatov et al., 2 Jun 2026).

A practical refinement is the revisit phase. After all trials, the algorithm selects the best completed trial, fixes its non-tree hyperparameters, and iteratively shifts left while the “excessive” condition $S_B$7 and $S_B$8 holds, returning the smallest $S_B$9 that preserves $B$00 (Porvatov et al., 2 Jun 2026). The paper notes, however, that this aggressive left-shift revisiting step can behave like a random walk with absorbing stop and may overshoot.

Parameter selection is tied to metric granularity. The tolerance $B$01 should not be smaller than the natural resolution of the empirical metric. For accuracy, a single-example change is approximately $B$02, giving the lower-bound heuristic

$B$03

For binary ROC-AUC, a single pair swap changes AUC by $B$04, giving

$B$05

The paper summarizes practical usage as $B$06, with the larger end more conservative and the smaller end pushing deeper into the plateau (Porvatov et al., 2 Jun 2026).

Empirically, across 12 datasets including the high-dimensional bioinformatics tasks Arcene and Dorothea, the selected number of trees differed substantially from common heuristics. For most classical benchmark datasets, PLATEAU selected fewer trees than fixed-range TPE or Hyperband with $B$07, whereas for Arcene and Dorothea it selected much larger $B$08, indicating that simple caps around $B$09–$B$10 can be insufficient in high-dimensional, low-density settings (Porvatov et al., 2 Jun 2026). Runtime with $B$11 was often lower than TPE or Hyperband with $B$12, although Diabetes, Arcene, and Dorothea were reported as exceptions. Joint optimization of the non-tree hyperparameters with the adaptive tree-count mechanism outperformed decoupled two-stage strategies in many cases, and increasing the trial budget from 40 to 120 materially improved best scores for both TPE and PLATEAU (Porvatov et al., 2 Jun 2026).

The stationary-distribution paper adds quantitative guidance for this regime. At $B$13, it reports $B$14, $B$15, and relative spread coefficient $B$16 for the modified rule, and $B$17, $B$18, and relative spread coefficient $B$19 for the original rule (Dukhovny et al., 29 Jun 2026). This implies that the leading standard deviation is about half of the stationary mean for both rules at $B$20, reinforcing that “practical convergence” should be interpreted as bounded fluctuation within a stationary band, not collapse to a single $B$21.

6. Relation to triplet-based local tracking and proposed cross-domain extension

The phrase “triplet-based” also appears in high-throughput particle tracking at LHCb, but there it denotes a different algorithmic object. "Search by triplet" is a local track-following method for the VELO detector, designed for SIMD/SIMT execution on CPUs and GPUs. Its implemented pipeline consists of three stages—Sort by phi, Triplet seeding and following, and the Tracklet filter—and it reconstructs effectively straight trajectories in the VELO region by exploiting local geometric continuity in $B$22 and a scatter proxy $B$23 (Pérez et al., 2022).

The tracking algorithm sorts hits by polar angle $B$24, mapped from $B$25 to uint16, uses binary search and pendulum search to collect up to $B$26 nearby candidates, forms triplets on consecutive modules, and follows them across later modules with one missing module allowed. Three-hit tracklets are subsequently validated by least-squares straight-line fits in $B$27-$B$28 and $B$29-$B$30 projections (Pérez et al., 2022). Its mathematical model writes

$B$31

with least-squares estimates for $B$32, residuals $B$33, $B$34, and a standard quality measure

$B$35

In this setting, the search-by-triplet scatter

$B$36

is a fast proxy to $B$37 used during candidate generation (Pérez et al., 2022).

The published paper is explicit that this VELO algorithm does not include an explicit plateau-search mechanism. Candidate selection relies on local geometric gates, the scatter score $B$38, best-hit selection per extension step, and flagging, with SIMT barriers enforcing consistency of the flags (Pérez et al., 2022). This directly distinguishes the implemented VELO method from the integrated triplet-based plateau-search algorithm of Random Forest tuning.

The same source nevertheless outlines how a plateau-search layer could be integrated into search by triplet. The proposed objective for a track hypothesis $B$39 with hits $B$40 is

$B$41

or, alternatively, a normalized $B$42 (Pérez et al., 2022). Local plateaus would be declared when an alternative hit changes $B$43 or the scatter proxy only marginally,

$B$44

The proposed integration replaces strict best-hit selection with a bounded-width beam over near-equivalent candidates, retains up to $B$45 triplets within $B$46 of the minimum during seeding, and extends tracks along all plateau candidates during following, subject to beam-width and budget constraints (Pérez et al., 2022).

Conflict resolution is likewise only proposed, not reported as deployed. Candidate hits may be reserved by multiple forming tracks during plateau exploration, with finalization resolving conflicts by minimizing $B$47 globally or by a tie-break such as longer track, smaller impact parameter, or lower $B$48; a local auction-style rule per module is suggested as an alternative to full global synchronization (Pérez et al., 2022). The paper’s stated performance target for such an integrated plateau search is to keep throughput within $B$49–$B$50 of the baseline at equal hardware and stream configuration while reducing clone fraction and maintaining low ghost rates. This suggests a conceptual parallel between the two domains: plateau search is being used to defer locally ambiguous commitments in a triplet-based process, but only the Random Forest formulation is presented as an implemented algorithm.

7. Limitations, misconceptions, and future directions

Several limitations are explicit in the Random Forest literature. The asymptotic theory relies on the tail model $B$51, the variance scaling $B$52, and Gaussian or centered folded-normal approximations for signed relative gaps; the stationary theory further assumes a factorized approximation for the two plateau tests at a given level and warm-start covariance scaling for nested forests (Dukhovny et al., 29 Jun 2026, Porvatov et al., 2 Jun 2026). The papers state that these approximations are asymptotic and may degrade for small $B$53, heavy-tailed score fluctuations, or strong dependencies beyond the nested approximation.

A frequent misconception is that plateau-specific pruning is only a computational shortcut. The Optuna/TPE formulation states that trials with $B$54 are pruned and do not contribute an objective value, so pruning acts as a quality filter, not merely acceleration (Porvatov et al., 2 Jun 2026). Another misconception is that a local plateau certificate determines the globally sufficient tree count exactly. The stationary-distribution results show instead that the search fluctuates around a stationary band and that the relative spread on the ensemble-size scale is independent of $B$55 to leading order (Dukhovny et al., 29 Jun 2026).

For the VELO extension, the limitations are architectural as well as algorithmic. The search-by-triplet paper identifies failure modes including very high occupancy events, overlapping or near-parallel tracks causing ambiguity in $B$56 windows, and noise hits producing spurious triplets; it notes that clone suppression via early flagging can occasionally discard legitimate alternatives in ambiguous regions (Pérez et al., 2022). The proposed plateau-search augmentation is correspondingly exposed to increased memory pressure, warp divergence, and contention when many tracks compete for the same hits. Validation would therefore need to track throughput, latency, occupancy or warp efficiency, memory bandwidth utilization, hit-assignment uniqueness, reproducibility, and the standard tracking metrics $B$57, $B$58, $B$59, $B$60, and $B$61 (Pérez et al., 2022).

Future directions in the Random Forest setting include adaptive $B$62 schedules, variance-aware step sizes using local estimates of $B$63, trajectory-based estimation of $B$64, $B$65, and $B$66, and multi-objective OOB criteria such as combinations of accuracy and calibration (Dukhovny et al., 29 Jun 2026). The Optuna/TPE paper further identifies integration with multi-fidelity HPO, application to other ensembles with monotone-in-budget behavior, and separate treatment for boosting, where overfitting makes score trajectories non-monotone (Porvatov et al., 2 Jun 2026). Taken together, these developments position the integrated triplet-based plateau-search algorithm as a triplet-local, tolerance-controlled, and explicitly stochastic methodology whose most mature instantiation is in Random Forest ensemble-size selection, while triplet-based particle tracking provides a technically related but distinct template for future plateau-aware search under severe parallel and real-time constraints.