--- title: Triplet-Based Plateau-Search Algorithm url: https://www.emergentmind.com/topics/integrated-triplet-based-plateau-search-algorithm type: topic --- # Triplet-Based Plateau-Search Algorithm Searching arXiv for the cited triplet-based plateau-search and search-by-triplet papers to ground the article in the relevant literature. The integrated triplet-based plateau-search algorithm is a triplet-driven adaptive procedure for selecting Random Forest ensemble size inside hyperparameter optimization by monitoring out-of-bag (OOB) score changes across a geometric triplet of forest sizes and shifting that triplet until a plateau is reached within a tolerance parameter. In the primary formulation, the number of trees is removed from the direct Optuna/TPE search space and is instead updated internally from trial to trial through local plateau tests on \(L=B/sf\), \(B\), and \(R=B\cdot sf\), with \(sf>1\) a fixed scale factor [2606.03549]. A later stationary-distribution analysis formalizes the central ensemble size as a stochastic process on a geometric grid rather than a deterministic stopping point, deriving equilibrium location and spread under a folded-normal approximation for OOB score differences [2606.30837]. A separate line of work on local particle tracking at LHCb also uses triplet-based search, but explicitly notes that the published "Search by triplet" algorithm does not include an explicit plateau-search mechanism; in that setting, plateau awareness is presented only as a proposed integration [2207.03936]. ## 1. Problem setting and defining features Random Forest tuning has a specific difficulty in selecting the number of trees because the predictive score typically improves monotonically with ensemble size, so classical HPO procedures such as Tree-structured Parzen Estimator and Hyperband require a predefined search interval \([T_{\min},T_{\max}]\) and often drive the estimate toward its right boundary [2606.03549]. Early-stopping strategies avoid fixing such a range, but can be sensitive to score noise and prone to premature stopping. The integrated triplet-based plateau-search algorithm addresses this by replacing direct search over `n_estimators` with a geometric triplet test on OOB scores and by updating the triplet adaptively across HPO trials [2606.03549]. The method is “integrated” in a precise sense. `n_estimators` is not sampled by TPE; instead, TPE samples the remaining Random Forest hyperparameters, while each trial trains three nested forests at \(L\), \(B\), and \(R\), computes their OOB scores, and applies a plateau test. Trials where no plateau is reached are pruned through plateau-specific pruning, and only trials that satisfy the right-plateau condition contribute an objective value [2606.03549]. This design uses information accumulated across trials while removing the ensemble size from the explicit Bayesian search space. The operational criterion is local and relative. With OOB scores \(S_L\), \(S_B\), and \(S_R\), the algorithm computes \[ \mathrm{platL}=\frac{|S_B-S_L|}{|S_B|},\qquad \mathrm{platR}=\frac{|S_R-S_B|}{|S_B|}. \] A tolerance \(\varepsilon>0\) defines whether the score change is still material or already within the plateau regime [2606.03549]. This gives a user-interpretable rule: \(B\) is treated as sufficient when the right-side relative change is no larger than \(\varepsilon\), and excessive when both sides satisfy the same bound. A central conceptual point is that the algorithm is not merely a stopping heuristic. The stationary-distribution formulation shows that after the remaining hyperparameters have stabilized, the central triplet point need not converge to a deterministic value; instead, it fluctuates around a stationary regime determined by \(\varepsilon\) and the scale factor [2606.30837]. This directly counters the common misconception that a plateau test must settle at a single terminal ensemble size. ## 2. Triplet mechanics and update rules The triplet is geometric rather than additive. For a current central ensemble size \(B\), the candidate sizes are \(\{B/\gamma, B, \gamma B\}\) in the stationary-distribution formulation and \([B/sf, B, B\cdot sf]\) in the Optuna/TPE formulation, with \(\gamma>1\) or \(sf>1\) fixed [2606.30837] [2606.03549]. The geometric spacing preserves a constant relative separation at large \(B\), whereas fixed additive steps would become indistinguishable as the ensemble grows. The decision logic is based on whether the left and right differences exceed the tolerance. The primary Optuna/TPE formulation uses three principal cases plus a noise-induced fourth case. The stationary-distribution paper gives both the original rule and a symmetric modified variant for the same triplet test [2606.30837]. | Plateau-test outcome | Original rule | Symmetric modified rule | |---|---|---| | \(d_L>\varepsilon,\ d_R>\varepsilon\) | shift right | shift right | | \(d_L>\varepsilon,\ d_R\le\varepsilon\) | stay | stay | | \(d_L\le\varepsilon,\ d_R\le\varepsilon\) | shift left | shift left | | \(d_L\le\varepsilon,\ d_R>\varepsilon\) | shift right | stay | With \(\Delta_t\in\{-1,0,+1\}\), the update is \[ B_{t+1}=\gamma^{\Delta_t}B_t. \] In the original rule, the mixed case is mapped to \(+1\), which the paper characterizes as conservative; in the symmetric modified rule, the same case is mapped to \(0\) [2606.30837]. In the Optuna/TPE implementation, the asymmetric case \(\mathrm{platL}\le\varepsilon\), \(\mathrm{platR}>\varepsilon\) is explicitly treated as shift right to counteract early stopping bias [2606.03549]. To avoid cumulative rounding, the implementation specifies explicit shift rules. Right shift updates \([L,B,R]\leftarrow[B,R,R\cdot sf]\); left shift updates \([L,B,R]\leftarrow[L/sf,L,B]\). In “stay” and “excessive” cases, the trial returns \(S_B\), favoring the smallest sufficient \(B\) because \(\mathrm{platR}\le\varepsilon\) guarantees \(S_R\approx S_B\) within tolerance [2606.03549]. This update structure makes the algorithm distinct from monotone sweep-based early stopping. Rather than scanning only to the right and halting at the first acceptable plateau, it allows right shifts, stays, and left shifts. A plausible implication is that the procedure can correct local underestimation and overestimation errors induced by OOB noise, which is exactly the phenomenon later formalized through the stationary regime [2606.30837]. ## 3. Stationary-distribution theory The stationary-distribution analysis models the grid index \(k\in\{0,1,2,\ldots\}\) through \[ B_k=B_0\gamma^k, \] and treats the evolving center \(B_t\) as a birth-death Markov chain on this geometric grid [2606.30837]. At level \(k\), the left- and right-gap plateau-pass probabilities are \[ \alpha_{L,k}=\mathbb{P}\!\left(\left|\frac{S_{B_k}-S_{B_k/\gamma}}{S_{B_k}}\right|\le\varepsilon\,\middle|\,D\right),\qquad \alpha_{R,k}=\mathbb{P}\!\left(\left|\frac{S_{B_k\gamma}-S_{B_k}}{S_{B_k}}\right|\le\varepsilon\,\middle|\,D\right). \] Under a factorized approximation for the two tests at the same level, the original rule uses \[ p_k^- \approx \alpha_{L,k}\alpha_{R,k},\qquad p_k^+ \approx 1-\alpha_{R,k},\qquad p_k^0 \approx \alpha_{R,k}(1-\alpha_{L,k}), \] while the modified rule uses \[ p_k^- \approx \alpha_{L,k}\alpha_{R,k},\qquad p_k^+ \approx (1-\alpha_{L,k})(1-\alpha_{R,k}), \] \[ p_k^0 \approx \alpha_{R,k}(1-\alpha_{L,k})+\alpha_{L,k}(1-\alpha_{R,k}). \] The stationary distribution \(\pi_k\) satisfies local balance \[ \pi_k\,p_k=\pi_{k+1}\,q_{k+1},\qquad \sum_{k\ge 0}\pi_k=1, \] with ratio form \[ \frac{\pi_{k+1}}{\pi_k}=\frac{p_k}{q_{k+1}}, \] and product-form masses \[ \pi_k=\pi_0\prod_{i=0}^{k-1}\frac{p_i}{q_{i+1}},\qquad k\ge 1. \] Under the paper’s variance asymptotics, \(\alpha_{L,k},\alpha_{R,k}\to 1\) as \(B_k\to\infty\), implying \(q_k\to 1\), \(p_k\to 0\), and therefore existence and uniqueness of a normalizable stationary distribution for any fixed \(\varepsilon>0\) [2606.30837]. The analytical approximation is built on a tail model for the conditional mean score, \[ \mu_T=S_\infty + cT^{-\gamma_{\text{score}}}+o(T^{-\gamma_{\text{score}}}), \] together with nested warm-start covariance scaling \[ \operatorname{Var}[S_T\mid D]\sim \frac{v}{T},\qquad \operatorname{Cov}[S_{T_1},S_{T_2}\mid D]\sim \frac{v}{T_2}\quad (T_10, \] and with \(R=sf\cdot B\), it proves \[ |S_\infty-S_B|\sim \frac{|S_R-S_B|}{1-sf^{-\gamma}}, \] and therefore, if \[ \left|\frac{S_R-S_B}{S_B}\right|\le \varepsilon, \] then \[ \left|\frac{S_\infty-S_B}{S_B}\right| \le \frac{\varepsilon}{1-sf^{-\gamma}}+o(1),\qquad B\to\infty. \] The observed local plateau gap thus controls the relative distance to the limiting score up to the multiplicative factor \((1-sf^{-\gamma})^{-1}>1\) [2606.03549]. The same paper derives an asymptotic variance estimate for the signed relative difference and for the absolute relative difference used by the algorithm. Assuming \((S_B,S_R)\mid D\) is approximately bivariate normal with means \((\mu_B,\mu_R)\), variances \((\sigma_B^2,\sigma_R^2)\), covariance \(\sigma_{BR}\), and \(\mu_B\ne 0\), \[ \operatorname{Var}\!\left[\frac{S_R-S_B}{S_B}\,\middle|\,D\right] \approx \frac{\mu_R^2}{\mu_B^4}\sigma_B^2+\frac{1}{\mu_B^2}\sigma_R^2-\frac{2\mu_R}{\mu_B^3}\sigma_{BR}. \] With \[ \sigma_B^2\sim \frac{v}{B},\qquad \sigma_R^2\sim \frac{v}{R},\qquad \sigma_{BR}\sim \frac{v}{R},\qquad B\to\infty, \] and \(\mu_B,\mu_R\to S_\infty\ne 0\), the asymptotic variance becomes \[ \operatorname{Var}\!\left[\frac{S_R-S_B}{S_B}\,\middle|\,D\right] \sim \left(\frac{v}{S_\infty^2}\right)\frac{1-sf^{-1}}{B}. \] For the absolute relative difference actually used in the triplet test, and under the tail model with \(\gamma>1/2\), \[ \operatorname{Var}\!\left[\left|\frac{S_R-S_B}{S_B}\right|\,\middle|\,D\right] \sim \left(1-\frac{2}{\pi}\right)\left(\frac{v}{S_\infty^2}\right)\frac{1-sf^{-1}}{B}, \qquad B\to\infty \] [2606.03549]. These formulas give the algorithm a direct statistical interpretation. OOB variability decays like \(O(B^{-1/2})\) in standard deviation, while the stationary center scales like \(O(\varepsilon^{-2})\). This suggests that smaller tolerances do not simply refine a deterministic solution; they move the stationary regime toward larger ensembles and retain a non-vanishing relative spread on the geometric grid [2606.30837] [2606.03549]. ## 5. Integration with Optuna/TPE and empirical behavior The practical implementation removes `n_estimators` from the Optuna search space and lets TPE sample only the remaining Random Forest hyperparameters, such as `max_features`, `max_depth`, `min_samples_leaf`, `min_samples_split`, and optionally the split criterion [2606.03549]. Each trial trains forests at \(L\), \(B\), and \(R\) using `warm_start=True`, so the total number of trees built per trial is \(R\), giving time complexity \(O(R)\) rather than \(O(L+B+R)\) [2606.03549]. The integrated implementation exposes three APIs: `tune_rf_oob()` as a classic TPE baseline with `n_estimators_range`, `tune_rf_oob_bohb()` as a Hyperband-like baseline, and `tune_rf_oob_plateau()` as the integrated triplet-based method [2606.03549]. The key parameters of `tune_rf_oob_plateau()` are `n_estimators_start = T0`, `scale_factor = sf`, `delta = \varepsilon`, and `max_trees`; the return values are a fitted Random Forest, an Optuna study, `best_n_estimators`, and a flag indicating whether any trial reached a plateau [2606.03549]. A practical refinement is the revisit phase. After all trials, the algorithm selects the best completed trial, fixes its non-tree hyperparameters, and iteratively shifts left while the “excessive” condition \(\mathrm{platL}\le\varepsilon\) and \(\mathrm{platR}\le\varepsilon\) holds, returning the smallest \(B\) that preserves \(\mathrm{platR}\le\varepsilon\) [2606.03549]. The paper notes, however, that this aggressive left-shift revisiting step can behave like a random walk with absorbing stop and may overshoot. Parameter selection is tied to metric granularity. The tolerance \(\varepsilon\) should not be smaller than the natural resolution of the empirical metric. For accuracy, a single-example change is approximately \(1/n\), giving the lower-bound heuristic \[ \varepsilon_{\text{acc}}\gtrsim \frac{1}{n\cdot acc_{\min}}. \] For binary ROC-AUC, a single pair swap changes AUC by \(1/(n_+n_-)\), giving \[ \varepsilon_{\text{auc}}\gtrsim \frac{1}{n_+n_-\cdot auc_{\min}}. \] The paper summarizes practical usage as \(\varepsilon\in[10^{-3},10^{-2}]\), with the larger end more conservative and the smaller end pushing deeper into the plateau [2606.03549]. Empirically, across 12 datasets including the high-dimensional bioinformatics tasks Arcene and Dorothea, the selected number of trees differed substantially from common heuristics. For most classical benchmark datasets, PLATEAU selected fewer trees than fixed-range TPE or Hyperband with \(T_{\max}\approx 2{,}565\), whereas for Arcene and Dorothea it selected much larger \(T\), indicating that simple caps around \(2{,}000\)–\(5{,}000\) can be insufficient in high-dimensional, low-density settings [2606.03549]. Runtime with \(\varepsilon=10^{-3}\) was often lower than TPE or Hyperband with \(T_{\max}=2{,}565\), although Diabetes, Arcene, and Dorothea were reported as exceptions. Joint optimization of the non-tree hyperparameters with the adaptive tree-count mechanism outperformed decoupled two-stage strategies in many cases, and increasing the trial budget from 40 to 120 materially improved best scores for both TPE and PLATEAU [2606.03549]. The stationary-distribution paper adds quantitative guidance for this regime. At \(\gamma=1.5\), it reports \(y_*^{\mathrm{mod}}\approx 0.744\), \(\sigma_{J,*}^{\mathrm{mod}}\approx 1.198\), and relative spread coefficient \(\log(\gamma)\sigma_{J,*}^{\mathrm{mod}}\approx 0.486\) for the modified rule, and \(y_*^{\mathrm{orig}}\approx 0.925\), \(\sigma_{J,*}^{\mathrm{orig}}\approx 1.299\), and relative spread coefficient \(\log(\gamma)\sigma_{J,*}^{\mathrm{orig}}\approx 0.526\) for the original rule [2606.30837]. This implies that the leading standard deviation is about half of the stationary mean for both rules at \(\gamma=1.5\), reinforcing that “practical convergence” should be interpreted as bounded fluctuation within a stationary band, not collapse to a single \(B\). ## 6. Relation to triplet-based local tracking and proposed cross-domain extension The phrase “triplet-based” also appears in high-throughput particle tracking at LHCb, but there it denotes a different algorithmic object. "Search by triplet" is a local track-following method for the VELO detector, designed for SIMD/SIMT execution on CPUs and GPUs. Its implemented pipeline consists of three stages—Sort by phi, Triplet seeding and following, and the Tracklet filter—and it reconstructs effectively straight trajectories in the VELO region by exploiting local geometric continuity in \(\phi\) and a scatter proxy \(S=\Delta x^2+\Delta y^2\) [2207.03936]. The tracking algorithm sorts hits by polar angle \(\phi\), mapped from \([-\pi,+\pi]\) to `uint16`, uses binary search and pendulum search to collect up to \(K\) nearby candidates, forms triplets on consecutive modules, and follows them across later modules with one missing module allowed. Three-hit tracklets are subsequently validated by least-squares straight-line fits in \(x\)-\(z\) and \(y\)-\(z\) projections [2207.03936]. Its mathematical model writes \[ x(z)=a_x z+b_x,\qquad y(z)=a_y z+b_y, \] with least-squares estimates for \((a_d,b_d)\), residuals \(r_{i,x}\), \(r_{i,y}\), and a standard quality measure \[ \chi^2=\sum_i \frac{r_i^2}{\sigma_i^2}. \] In this setting, the search-by-triplet scatter \[ S=\Delta x^2+\Delta y^2 \] is a fast proxy to \(r_i^2\) used during candidate generation [2207.03936]. The published paper is explicit that this VELO algorithm does not include an explicit plateau-search mechanism. Candidate selection relies on local geometric gates, the scatter score \(S\), best-hit selection per extension step, and flagging, with SIMT barriers enforcing consistency of the flags [2207.03936]. This directly distinguishes the implemented VELO method from the integrated triplet-based plateau-search algorithm of Random Forest tuning. The same source nevertheless outlines how a plateau-search layer could be integrated into search by triplet. The proposed objective for a track hypothesis \(\tau\) with hits \(\{i\}\) is \[ J(\tau)=\sum_i w_i r_i^2 = \sum_i w_i\left[(x_i-(a_x z_i+b_x))^2+(y_i-(a_y z_i+b_y))^2\right], \] or, alternatively, a normalized \(\chi^2\) [2207.03936]. Local plateaus would be declared when an alternative hit changes \(J\) or the scatter proxy only marginally, \[ \Delta J = J(\tau\oplus h')-J(\tau\oplus h)\le \delta, \qquad \Delta S = S(h')-S(h)\le \delta_S. \] The proposed integration replaces strict best-hit selection with a bounded-width beam over near-equivalent candidates, retains up to \(B\) triplets within \(\Delta S\le\delta_S\) of the minimum during seeding, and extends tracks along all plateau candidates during following, subject to beam-width and budget constraints [2207.03936]. Conflict resolution is likewise only proposed, not reported as deployed. Candidate hits may be reserved by multiple forming tracks during plateau exploration, with finalization resolving conflicts by minimizing \(J\) globally or by a tie-break such as longer track, smaller impact parameter, or lower \(\chi^2\); a local auction-style rule per module is suggested as an alternative to full global synchronization [2207.03936]. The paper’s stated performance target for such an integrated plateau search is to keep throughput within \(0.7\)–\(0.9\times\) of the baseline at equal hardware and stream configuration while reducing clone fraction and maintaining low ghost rates. This suggests a conceptual parallel between the two domains: plateau search is being used to defer locally ambiguous commitments in a triplet-based process, but only the Random Forest formulation is presented as an implemented algorithm. ## 7. Limitations, misconceptions, and future directions Several limitations are explicit in the Random Forest literature. The asymptotic theory relies on the tail model \(S_T=S_\infty+cT^{-\gamma}+o(T^{-\gamma})\), the variance scaling \(\sigma_B^2\sim v/B\), and Gaussian or centered folded-normal approximations for signed relative gaps; the stationary theory further assumes a factorized approximation for the two plateau tests at a given level and warm-start covariance scaling for nested forests [2606.30837] [2606.03549]. The papers state that these approximations are asymptotic and may degrade for small \(B\), heavy-tailed score fluctuations, or strong dependencies beyond the nested approximation. A frequent misconception is that plateau-specific pruning is only a computational shortcut. The Optuna/TPE formulation states that trials with \(\mathrm{platR}>\varepsilon\) are pruned and do not contribute an objective value, so pruning acts as a quality filter, not merely acceleration [2606.03549]. Another misconception is that a local plateau certificate determines the globally sufficient tree count exactly. The stationary-distribution results show instead that the search fluctuates around a stationary band and that the relative spread on the ensemble-size scale is independent of \(\varepsilon\) to leading order [2606.30837]. For the VELO extension, the limitations are architectural as well as algorithmic. The search-by-triplet paper identifies failure modes including very high occupancy events, overlapping or near-parallel tracks causing ambiguity in \(\phi\) windows, and noise hits producing spurious triplets; it notes that clone suppression via early flagging can occasionally discard legitimate alternatives in ambiguous regions [2207.03936]. The proposed plateau-search augmentation is correspondingly exposed to increased memory pressure, warp divergence, and contention when many tracks compete for the same hits. Validation would therefore need to track throughput, latency, occupancy or warp efficiency, memory bandwidth utilization, hit-assignment uniqueness, reproducibility, and the standard tracking metrics \(\varepsilon\), \(f_{\text{fake}}\), \(f_{\text{clone}}\), \(p_{\text{hit}}\), and \(\varepsilon_{\text{hit}}\) [2207.03936]. Future directions in the Random Forest setting include adaptive \(\gamma\) schedules, variance-aware step sizes using local estimates of \(s(B)\), trajectory-based estimation of \(\pi_k\), \(B_*\), and \(\sigma_{B,*}\), and multi-objective OOB criteria such as combinations of accuracy and calibration [2606.30837]. The Optuna/TPE paper further identifies integration with multi-fidelity HPO, application to other ensembles with monotone-in-budget behavior, and separate treatment for boosting, where overfitting makes score trajectories non-monotone [2606.03549]. Taken together, these developments position the integrated triplet-based plateau-search algorithm as a triplet-local, tolerance-controlled, and explicitly stochastic methodology whose most mature instantiation is in Random Forest ensemble-size selection, while triplet-based particle tracking provides a technically related but distinct template for future plateau-aware search under severe parallel and real-time constraints. Source: https://www.emergentmind.com/topics/integrated-triplet-based-plateau-search-algorithm