Tomographic Quantile Forests (TQF)

• Tomographic Quantile Forests are a nonparametric, tree-based regression method that estimates multivariate conditional distributions via directional quantile projections.
• It leverages the Cramér–Wold theorem and minimizes the sliced Wasserstein distance to capture arbitrary, nonconvex, and multimodal support shapes.
• The framework integrates an augmented QRF++ model with alternating convex optimization to deliver efficient, uncertainty-quantified predictions in multivariate regression.

Tomographic Quantile Forests (TQF) are a nonparametric, tree-based regression approach for uncertainty-quantified prediction in multivariate response problems, designed to learn and reconstruct the full conditional distribution $P(y \mid x)$ of a vector-valued target using quantile estimation along arbitrary directions. TQF leverages all one-dimensional projections of the response, invoking the Cramér–Wold theorem to uniquely determine multivariate conditional laws, and reconstructs these distributions via sliced Wasserstein distance minimization. The framework integrates an augmented quantile forest model (“QRF++”) for efficient directional quantile regression with an alternating convex optimization procedure for distributional reconstruction, enabling flexible, nonconvex, and multimodal uncertainty representation without separately training models for each direction (Kanazawa, 18 Dec 2025).

1. Multivariate Regression and Problem Setting

Given covariates $x \in \mathbb{R}^p$ and multivariate responses $y \in \mathbb{R}^d$, the objective is to recover the conditional law $P(y \mid x)$, not merely its mean or marginal characteristics. TQF exploits the mathematical property that the laws of all projected variables $u = n^\top y$ for $n \in S^{d-1}$, the unit sphere in $\mathbb{R}^d$, determine the law of $y$ through the Cramér–Wold device. This setup translates the multivariate distributional estimation problem into a continuum of one-dimensional quantile regression tasks, establishing the foundation for conditional distribution learning in arbitrary directions.

2. Directional Quantile Estimation in TQF

The key modeling step is learning the conditional $\tau$-quantile function $Q_\tau(n^\top y \mid x)$ for every direction $n$ and quantile level $\tau \in (0,1)$, defined as:

$$Q_\tau(n^\top y \mid x) = \inf\{q : \Pr[n^\top y \leq q \mid x] \geq \tau\}.$$

Training aims to minimize the pinball (check) loss,

$$\rho_\tau(r) = \begin{cases} \tau r, & r \geq 0 \ (\tau-1) r, & r < 0 \end{cases},$$

across augmented input data where directional and Fourier features are incorporated to capture all-projection dependence and higher-order distributional features.

TQF adopts the QRF++ backbone—an extension of Quantile Regression Forests—embedding $(x, n)$ as the input and using output targets comprising multiple quantiles and random Fourier features of $u = n^\top y$. In training, for each data pair, $G$ independent random projection directions are sampled, and orthogonal rotations further augment the feature set. This produces $G \times N$ records per sample, supporting tree-based partitioning that is sensitive to input and projection configuration.

Model symmetrization ensures the quantile function satisfies

$$Q_\tau(n^\top y \mid x) = -Q_{1-\tau}(-n^\top y \mid x)$$

by defining the symmetrized model

$$F_Q^*(x, n, \tau) = \frac{1}{2} [F_Q(x, n, \tau) - F_Q(x, -n, 1-\tau)],$$

where $F_Q$ is the raw quantile prediction forest.

3. Multivariate Distribution Reconstruction via Sliced Wasserstein Matching

After the forest is trained, for a test covariate $x_\mathrm{new}$, quantiles are predicted for each of $K$ randomly chosen directions $\{n_k\}_{k=1}^K$ at $M$ quantile levels:

$$\{Q_{k, m}\} = F_Q^*(x_\mathrm{new}, n_k, q_m), \quad m = 1, \ldots, M.$$

The reconstruction task is then cast as finding a discrete empirical measure $Z = \{(w_j, z_j)\}_{j=1}^J$ approximating $P(y \mid x_\mathrm{new})$ by minimizing the (discrete) sliced 1-Wasserstein loss:

$$L(Z) = \frac{1}{K}\sum_{k=1}^K\frac{1}{M} \sum_{m=1}^M |Q(n_k, q_m; Z) - Q_{k, m}|,$$

where $Q(n_k, q_m; Z)$ denotes the $q_m$-quantile of the projection $n_k^\top y$ under $Z$. The optimization alternates:

• Weight step: Solve the convex optimization of weights $\{w_j\}$ for fixed supports,
• Support step: Fit KDE to current weighted cloud and resample supports,
• Ensemble merging: Run parallel alternations, combine supports, and prune low-weight or redundant points via loss minimization.

The process is summarized in the following pseudocode:

Input: D_slice = { (n_k, q_m, Q_{k,m}) } for k=1..K, m=1..M N0 = initial support size, N1 = regular support size, E = ensemble size Output: Weighted point cloud Z_merged 1. Initialize z_j (j=1..N0) 2. Uniform weights w_j ← 1/N0 3. Optimize {w_j} to minimize L({(w_j, z_j)}, D_slice) 4. Repeat until convergence: a. Fit KDE to current cloud Z b. Sample N1 new {z_j} from KDE c. Optimize weights {w_j} on new support d. Update loss, check for decrease 5. For e=1..E in parallel: a. Sample N1 from final KDE → {z_j^(e)} b. Optimize weights {w_j^(e)} on {z_j^(e)} c. Collect all (w_j^(e), z_j^(e)) into Z_* 6. Prune Z_*: For ℓ, keep top ℓ points, renormalize, compute L_ℓ Choose ℓ* minimizing L_ℓ Return top ℓ* points

This Quantile-Matching Empirical Measure (QMEM) procedure efficiently yields a support-weighted point cloud representing the learned $P(y \mid x)$.

4. Computational Complexity and Algorithmic Characteristics

The main training phase requires tree induction for $B$ trees across $G \times N$ samples, yielding $O(B G N \log(GN))$ complexity per coordinate due to ensemble-based splitting. Quantile predictions require $O(B K \log(GN))$ operations for $K$ query directions.

QMEM reconstruction involves a convex optimization in $J$ variables ($O(J^2)$ per solve, or $O(J^3)$ in the worst case), repeated over $I_\mathrm{alt}$ alternations and $E$ ensembles. With practical settings $J\sim 1000$ and $E\sim 20$, this results in manageable per-query cost.

Typically, $K \in [30, 100]$ directions suffice for $d \leq 5$; in higher dimensions, quasi-Monte Carlo sampling or spherical designs may be employed for improved projection coverage.

5. Theoretical Properties and Methodological Comparisons

TQF’s QMEM stage produces an empirical measure minimizing the sliced-1-Wasserstein distance to the model-predicted projected quantiles. The sliced Wasserstein loss is convex with respect to the weights, with global convergence and stability under gradient-based optimization.

Under standard forest honesty and sufficient data (with $N \to \infty$ and $G, K, M \to \infty$), consistency holds for the estimated directional quantiles, and the QMEM reconstruction converges in sliced Wasserstein distance.

Classical Directional Quantile Regression (DQR) fits separate (typically linear) models for each direction, intersecting quantile-defining halfspaces to obtain only convex central regions, which cannot represent nonconvex or multimodal conditional supports. TQF overcomes these limitations by modeling all directions simultaneously via a nonparametric forest, imposing no convexity or unimodality restriction, and capturing arbitrary support shapes (e.g., two moons, annuli, regions with holes). TQF thus generalizes DQR approaches by enabling efficient joint estimation and reconstruction without restrictive assumptions.

6. Practical Aspects and Implementation Guidance

Quantile regions at level $\tau$ and direction $n$ are obtained as the halfspace $\{y: n^\top y \leq F_Q^*(x, n, \tau)\}$. Central conditional regions emerge as intersections over multiple random directions.

To sample from the reconstructed $\hat P(y \mid x)$, one simply draws a support point $z_j$ with probability $w_j$ from the final QMEM, optionally adding Gaussian noise for smoothness.

Essential hyperparameters include:

• $B$ (trees in forest): 50–200
• $T$ (number Fourier frequencies): 0–10
• $G, \tilde G$ (sample/feature augmentations): 5–20
• $M, K$ (quantile and projection discretizations): 20–50
• $N_0, N_1$ (support sizes for QMEM): $N_0 \sim 10$, $N_1 \sim 100$
• $E$ (ensemble runs in QMEM): 10–20

Larger $G, \tilde G$ improve directional dependence capture but increase training cost. $M$ controls granularity of quantile reconstruction, with higher values resolving finer features. $K$ sets a tradeoff between reconstruction fidelity and speed. $N_1$ should balance expressivity for multimodal supports against computational burden in the convex optimization of weights.

TQF presents a framework for nonparametric, parallelizable, and distribution-free multivariate conditional uncertainty estimation, suited primarily to tabular data settings and providing an unrestricted, data-adaptive characterization of predictive uncertainty (Kanazawa, 18 Dec 2025).