Forest-Proximity Preconditioning in SBI

• Forest-proximity preconditioning is a weighting scheme for simulation-based inference that assigns high weights to simulated summaries consistently co-occurring in regression-forest leaves.
• It reduces the amortisation gap by replacing the global training distribution with a locally weighted version, enhancing robustness in neural posterior estimation under model misspecification.
• Empirical findings show that combining forest-proximity preconditioning with robust neural posterior estimation improves stability, calibration, and posterior-predictive fit compared to traditional methods.

Forest-proximity preconditioning is a summary-space adaptive weighting scheme for simulation-based inference that uses tree-based proximity scores to concentrate learning around an observed dataset. It was introduced within preconditioned robust neural posterior estimation (PRNPE) for misspecified simulators, where incompatible summaries and extreme prior-predictive simulations can make standard neural posterior estimation unreliable through extrapolation and wasted training capacity (Kelly et al., 20 Feb 2026). In the narrow sense, the method assigns high weight to simulated summaries that repeatedly co-occur with the observed summary in regression-forest leaves, then trains posterior and summary-density estimators on the resulting localized sample. In a broader bibliographic sense, the phrase touches adjacent literatures on forest proximities in time-series learning and forest-based preconditioners for graph optimization, but those are distinct constructions rather than the same method.

1. Terminology and Delimitation

In current research usage, “forest-proximity preconditioning” most directly denotes the PRNPE weighting mechanism built from random-forest leaf co-occurrence in summary space. Closely related phrases appear in two other technical contexts. One is PF-GAP, which extends RF-GAP proximities to proximity forests for time series and is presented as a supervised representation-learning or preconditioning step for downstream tasks. The other is combinatorial preconditioning for proximal graph optimization, where graph edges are partitioned into forests to define a variable metric with favorable condition numbers. These uses share the words forest and either proximity or preconditioning, but they operate on different objects and optimize different targets (Shaw et al., 2024, Möllenhoff et al., 2018, Ye et al., 2020).

The phrase should also be distinguished from astrophysical work on quasar proximity zones measured with the Lyman-$\alpha$ forest. There, “forest” denotes absorption structure in quasar spectra and “proximity” denotes the balance between excess ionization and gas overdensity near quasars, not an algorithmic weighting or conditioning procedure (Hada et al., 20 Dec 2025).

Usage	Forest object	Functional role
Forest-proximity preconditioning	Regression-forest leaves in summary space	Localize SBI training near $\mathbf{s}_y$
PF-GAP	Proximity-forest / RF-GAP tree structure	Embeddings, outlier analysis, representation learning
Combinatorial forest preconditioning	Forest decomposition of graph edges	Block-diagonal metric for PG/PDHG
Quasar proximity in the Lyman-$\alpha$ forest	Spectral absorption forest	Observational probe of quasar environments

2. Statistical Setting and Motivation

The PRNPE formulation considers a simulator model with prior, likelihood-free simulator, and summaries

$$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$

with observed summary

$$\mathbf{s}_y = S(\mathbf{y}), \qquad \pi(\boldsymbol{\theta}\mid \mathbf{s}_y).$$

The central difficulty is model misspecification at the summary level, formalized by

$$\epsilon^* = \inf_{\boldsymbol{\theta}\in\Theta}\rho\!\left(b_\star, b(\boldsymbol{\theta})\right) > 0.$$

Under this regime, observed summaries may lie in a low-support or even outside-support region of the simulator’s training distribution, while prior-predictive draws can contain extreme summaries that dominate optimization (Kelly et al., 20 Feb 2026).

Standard neural posterior estimation trains a conditional density estimator $$q_{\boldsymbol{\phi}}(\boldsymbol{\theta}\mid \mathbf{s})$$ by minimizing a global risk over the training distribution,

$$\mathcal{L}(\boldsymbol{\phi}) = \mathbb{E}_{(\boldsymbol{\theta},\mathbf{s})\sim p_{\text{train}}(\boldsymbol{\theta},\mathbf{s})} \big[-\log q_{\boldsymbol{\phi}}(\boldsymbol{\theta}\mid \mathbf{s})\big].$$

For a single observation, however, the quantity of interest is the local risk at $\mathbf{s}_y$,

$$\mathcal{L}_{\mathbf{s}_y}(\boldsymbol{\phi}) = \mathbb{E}_{\boldsymbol{\theta}\sim \pi(\cdot\mid \mathbf{s}_y)} \big[-\log q_{\boldsymbol{\phi}}(\boldsymbol{\theta}\mid \mathbf{s}_y)\big].$$

The mismatch between these objectives is the amortisation gap,

$\mathbf{s}_y$0

where $\mathbf{s}_y$1 is the best achievable local log-loss within the model family. Preconditioning addresses this mismatch by replacing the training distribution with a weighted version,

$\mathbf{s}_y$2

and the paper gives a bound showing that the amortisation gap is controlled by weighted distances to $\mathbf{s}_y$3:

$\mathbf{s}_y$4

A basic property is conditional invariance:

$\mathbf{s}_y$5

Thus the weighting changes which simulations are emphasized without changing the conditional target on the retained support. This is the formal rationale for treating preconditioning as localization rather than posterior deformation.

3. Construction of the Forest-Proximity Weights

The method constructs weights from regression forests, borrowing its logic from ABC random forests. For each parameter coordinate $\mathbf{s}_y$6, with $\mathbf{s}_y$7, one regression forest is fitted. In each tree $\mathbf{s}_y$8 of forest $\mathbf{s}_y$9, the leaf containing the observation is denoted

$\alpha$0

A simulated summary $\alpha$1 receives weight according to how often it falls into the same leaves as the observation:

$\alpha$2

The weights are then normalized as

$\alpha$3

Large weights therefore correspond to repeated leaf co-occurrence with $\alpha$4, while the denominator makes smaller leaves contribute more strongly than larger ones (Kelly et al., 20 Feb 2026).

The paper characterizes this construction as a data-adaptive kernel on summary space. It is tolerance-free, unlike ABC thresholds, and it requires no explicit discrepancy $\alpha$5 and no explicit distance metric. The stated purpose is to assign large weights to simulations with summaries similar to $\alpha$6, small weights to outlying or pathological simulations, and to do so adaptively from the forest geometry. To make the procedure conservative, the authors include all summary features at every split, constrain tree depth, and use large minimum leaf sizes. This is intended to filter extremes aggressively while retaining enough effective sample size. A plausible implication is that the method trades metric specification for a learned notion of neighborhood induced by repeated partitioning of summary space.

4. Role within Preconditioned Robust Neural Posterior Estimation

Forest-proximity preconditioning is not used in isolation; it is the localization stage of PRNPE. The pipeline begins with prior-predictive simulation,

$\alpha$7

Weights are computed as $\alpha$8, normalized, and then used to resample indices

$\alpha$9

forming a preconditioned dataset. The summaries $$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$0 and $$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$1 are subsequently standardized using weighted mean and standard deviation from the weighted sample (Kelly et al., 20 Feb 2026).

Training then proceeds in two weighted stages. The conditional posterior estimator is fit by

$$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$2

and the summary marginal estimator by

$$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$3

Robustness is then supplied by robust neural posterior estimation, which augments NPE with an explicit error model for mismatch between simulated and observed summaries. In the paper’s formulation, RNPE learns a conditional posterior estimator, a marginal density estimator, and a spike-and-slab error model for $$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$4. Latent denoised summaries are drawn from

$$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$5

and posterior samples are averaged according to

$$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$6

Within this architecture, forest-proximity preconditioning localizes the training distribution, while RNPE absorbs summary incompatibility through the latent denoising stage. This suggests that the method is designed as a complement to robustness rather than a replacement for it.

5. Empirical Findings

Across two synthetic examples and one real example, the reported result is that preconditioning combined with robust NPE improves stability, calibration, accuracy, and posterior-predictive fit over standard baseline methods. The paper emphasizes that standard NPE and RNPE alone can perform poorly or be unstable under incompatible summaries and extreme prior-predictive behavior, whereas non-robust preconditioned baselines are generally weaker than the robust versions (Kelly et al., 20 Feb 2026).

Example	Forest-proximity PRNPE results	Comparative note
Contaminated Weibull	RMSE 0.07; posterior predictive distance -0.62; coverage 0.85	SMC-ABC PRNPE: 0.09, -0.53, 0.98
Sparse VAR	Bias 0.01; RMSE 0.02; Coverage 0.98	SMC-ABC PRNPE was best overall; forest-proximity PRNPE was nearly as good
Real pancreatic tumour model (BVCBM)	Best posterior predictive fit on most datasets; strongest on D1–D3; tied best on D4	NPE and RNPE were unstable on some datasets

These results place the method in a nuanced position. In the Contaminated Weibull example, forest-proximity PRNPE achieved lower RMSE and better posterior predictive distance than SMC-ABC PRNPE, but its coverage was lower. In Sparse VAR, the forest-proximity variant was competitive rather than dominant. In the pancreatic tumour application, the strongest gains were in posterior predictive fit, not necessarily in every inference metric. The evidence therefore supports the method as an effective localization mechanism under misspecification, but not as a uniformly superior replacement for all preconditioning strategies.

6. Comparison with Other Preconditioning Schemes

The main benchmark in the paper is SMC-ABC preconditioning. In that scheme, a short ABC or SMC-ABC warm-up is used, requiring an explicit discrepancy $$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$7 and tolerances, and in the earlier framework the resulting ABC samples serve as a proposal for later rounds. Forest-proximity preconditioning differs in four stated respects: it uses random-forest leaf co-occurrence, it is tolerance-free, it does not require manual specification of a distance, and it works directly on prior-predictive simulations to build weights (Kelly et al., 20 Feb 2026).

A further distinction concerns how the conditional estimator is trained. In the forest-proximity construction, the conditional posterior estimator is trained directly on the preconditioned sample; the method does not fit an intermediate unconditional density before returning to posterior learning. Because the weights depend only on summaries, the paper argues that NPE remains well-posed through the conditional-invariance property. By contrast, for neural likelihood estimation the weighted likelihood becomes

$$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$8

so additional correction is needed. This gives forest-proximity preconditioning a method-specific compatibility with NPE-style objectives.

The paper’s empirical comparisons also clarify the method’s limitations. Forest-proximity weighting is especially aimed at down-weighting extreme prior-predictive simulations and concentrating computation around the observed dataset; it is not, by itself, the robustness mechanism. The improvement reported in the paper occurs when localization is paired with RNPE’s explicit error model. This suggests that forest-proximity preconditioning is best interpreted as a summary-space conditioning device whose effectiveness depends on the broader inference stack in which it is embedded.

7. Adjacent Research Lines

A neighboring but distinct literature studies forest proximities as geometry-preserving representations for time series. PF-GAP extends RF-GAP to proximity forests by modifying the original proximity-forest implementation to use bootstrap sampling, thereby creating the in-bag and out-of-bag structure required by GAP-style proximities. The method symmetrizes generally asymmetric proximities, converts them to dissimilarities, and applies metric or non-metric MDS. On three UCR datasets, the resulting DGAP dissimilarity achieved the best $$\pi(\boldsymbol{\theta}), \qquad p(\mathbf{x}\mid \boldsymbol{\theta}), \qquad \mathbf{s}=S(\mathbf{x}),$$9-means clustering score in every case—1.0 on GunPoint, 0.94 on ItalyPowerDemand, and 0.75 on ArrowHead—and the best F1 scores for the association between misclassification and outlier status—1.0, 0.99, and 0.99, respectively (Shaw et al., 2024). That paper explicitly presents PF-GAP as a supervised representation-learning or preconditioning step, but the object being preconditioned is downstream geometric analysis, not likelihood-free posterior learning.

A second adjacent line concerns forest-based combinatorial preconditioners for graph optimization. There, the graph edge set is partitioned into forests,

$$\mathbf{s}_y = S(\mathbf{y}), \qquad \pi(\boldsymbol{\theta}\mid \mathbf{s}_y).$$0

and a block-diagonal metric is formed from forest-local operators,

$$\mathbf{s}_y = S(\mathbf{y}), \qquad \pi(\boldsymbol{\theta}\mid \mathbf{s}_y).$$1

This construction reduces the condition number of the scaled operator while keeping the dual proximal step tractable because each block corresponds to a forest. For regular $$\mathbf{s}_y = S(\mathbf{y}), \qquad \pi(\boldsymbol{\theta}\mid \mathbf{s}_y).$$2D grids, the horizontal/vertical chain decomposition yields

$$\mathbf{s}_y = S(\mathbf{y}), \qquad \pi(\boldsymbol{\theta}\mid \mathbf{s}_y).$$3

independent of grid size (Möllenhoff et al., 2018). An adaptive extension based on active-set analysis later showed that nested-forest decomposition of the inactive edges yields a guaranteed local linear convergence rate, with practical reconditioning implemented by a greedy heuristic built from edge weights

$$\mathbf{s}_y = S(\mathbf{y}), \qquad \pi(\boldsymbol{\theta}\mid \mathbf{s}_y).$$4

and repeated minimum spanning forest extraction (Ye et al., 2020).

Taken together, these adjacent literatures show that “forest-proximity preconditioning” is part of a broader methodological pattern in which forests encode tractable local structure. In SBI, that structure defines adaptive neighborhoods in summary space; in time-series learning, it induces supervised proximities and embeddings; in graph optimization, it defines scalable variable metrics for proximal solvers. The shared motif is not a single algorithm but the use of forest-induced geometry to improve conditioning, localization, or representation.