Super-Level-Set Regression: Conditional Quantiles via Volume Minimization

Abstract: Constructing minimum-volume prediction regions that satisfy conditional coverage is a fundamental challenge in multivariate regression. Standard approaches rely on explicitly estimating the full conditional density and subsequently thresholding it. This two-step plug-in process is notoriously difficult, sensitive to estimation errors, and computationally expensive. One would like to instead optimize the region directly. Formulating a direct solution is challenging, however, because it requires minimizing a volume objective that is coupled with the conditional quantiles of the model's own estimation error. In this work, we address this challenge. We introduce super-level-set regression (SLS), a novel mathematical framework that successfully resolves this implicit coupling, allowing us to directly parameterize and optimize the geometric boundaries of the target conditional level sets. By bypassing full distribution estimation and leveraging flexible volume-preserving frontier functions, our approach natively captures complex, multimodal, and disjoint conditional structures end-to-end. Ultimately, SLS offers a new perspective on multivariate conditional quantile regression, replacing the restrictive assumptions of density-first methods with a direct geometric optimization strategy.

• The paper introduces SLS regression, a direct optimization method that minimizes the volume of prediction sets to achieve conditional quantile coverage without explicit density estimation.
• It leverages volume-preserving neural flows and a shrinking-window surrogate loss to adaptively model complex, multimodal, and heavy-tailed distributions.
• Empirical results on both synthetic and real-world datasets demonstrate that SLS regression yields compact, robust regions that outperform traditional density-based approaches.

Super-Level-Set Regression: Conditional Quantiles via Direct Volume Minimization

Introduction and Context

The computation of conditional level sets that capture a prescribed probability mass is essential for robust uncertainty quantification in multivariate regression. Classical solutions operate in two distinct steps: (i) nonparametric estimation of the full conditional density, followed by (ii) thresholding to extract highest density regions (HDRs) at the desired coverage level. This plug-in route is statistically and computationally challenging, especially in high dimensions or when $Y \mid X$ is multimodal, heavy-tailed, or otherwise structurally complex.

The paper "Super-Level-Set Regression: Conditional Quantiles via Volume Minimization" (2605.06210) proposes a significant methodological shift, recasting the problem as the direct minimization of region volume under a hard conditional coverage constraint. The authors introduce a framework—Super-Level-Set (SLS) regression—that avoids density estimation entirely, instead parameterizing regions via data-dependent frontier functions and optimizing their geometry to achieve minimal volume at the prescribed quantile.

Problem Formulation and Methodology

The SLS framework seeks, for a covariate vector $X$, to estimate a set $\mathcal{A}(X)$ of minimal Lebesgue measure (volume) subject to $\mathbb{P}(Y \in \mathcal{A}(X) \mid X) \geq \tau$. Explicit density estimation is bypassed by introducing a parameterized family of frontier functions $G$, which define sublevel sets in joint $(X, Y)$ space. The constrained optimization thus formally becomes:

$$\inf_{G,\,q} \mathbb{E}_X[\text{Vol}_G(q(X), X)] \quad\text{s.t.}\quad q(X) \geq \text{Quantile}_\tau(G(X, Y)\mid X)$$

where $\text{Vol}_G(q(X),X)$ is the volume of the sublevel set $\{y : G(X,y)\leq q(X)\}$.

Critically, this problem involves a nontrivial bi-level coupling: $G$ defines both the set geometry and the quantile constraint. Standard approaches for quantile optimization (differentiable sorting, soft Lagrangians, smoothed relaxations, or Conditional Value-at-Risk [CVaR]) either do not differentiate through conditional quantiles or relax coverage to marginal control, thus failing to address the true conditional guarantee.

The authors resolve this by introducing a surrogate loss: instead of minimizing only at fixed $X$0, they optimize an average of the set volume over a small probability band $X$1—a "shrinking window"—ensuring that as training proceeds and $X$2, the surrogate converges tightly to the desired quantile. This averaging approach preserves differentiability and is theoretically justified by a uniform convergence guarantee, as formally shown in Proposition 1.

Geometric Parameterization and Expressivity

Frontier functions $X$3 are constructed to allow efficient, closed-form volume calculation and flexible boundary modeling. For unimodal or near-elliptical distributions, a flow-based Mahalanobis frontier is used:

$X$4

$X$5 is a conditional, volume-preserving normalizing flow, guaranteeing that all sets of constant $X$6 have known volume regardless of the underlying data geometry. This parameterization, through the use of expressive neural flows and location/shape adaptation, allows SLS to target highly flexible sets, including complex, non-convex, and multimodal regions.

The topological support of SLS is further broadened using a union of $X$7 independent flows, aggregated through a soft minimum. This enables the modeling of disconnected and intricate multimodal sets—a key limitation of diffeomorphism-based frontiers and directional quantile methods.

Optimization Strategy

The SLS algorithm alternates between updating (i) the frontier parameters to minimize the average volume over the quantile window, and (ii) the quantile parameters via a pinball loss, ensuring proper tracking of the conditional quantile for the current $X$8. This alternating, annealed optimization is rigorously justified by the convergence result.

A warm-start phase—an unweighted global average volume minimization—is used to avoid degenerate early solutions. The subsequent shrinking window phase focuses the optimization around the target quantile, correcting for cases where global minimization leads to suboptimal region placement (notably in asymmetric or skewed settings).

Empirical Results

Synthetic Scenarios

SLS regression is extensively validated on synthetic data (Figures 1, 2, 3, 5), demonstrating robustness across diverse conditional distributions:

• In univariate and low-dimensional scenarios, SLS can correctly adapt to skewness introduced by asymmetric noise (Figure 1, left), aligning region centerings precisely at the quantile of interest and not the mean or median. Notably, the shrinking window mechanism is essential for correcting the bias introduced by initialization.

Figure 1: For an asymmetric exponential, the windowed optimization shifts the region off the conditional median, ensuring true coverage/volume optimality.

• For distributions with transitioning or strongly multimodal conditional structure, the union-of-flows frontier enables topologically adaptive set estimation, with components activating selectively as required by local data geometry (Figure 1, right).

Figure 1: In a conditional bimodal regime, SLS with K flows morphs the region topology between unimodal and disjoint intervals as covariate $X$9 varies.

• Under heavy-tailed or contaminated noise, as in outlier scenarios, the direct volume minimization of SLS entirely discounts low-density, irrelevant mass (Figure 2, right), in contrast to density estimation-based methods whose region size is inflated by outlier modeling.

Figure 2: SLS recovers the minimum-volume, high-density region even in the presence of adversarial outliers, where non-robust likelihood approaches fail.

Real Data

Benchmarking on multivariate real datasets (Figure 3) against state-of-the-art baselines confirms that SLS regression yields efficient regions (smaller average scaled volume) with competitive or superior conditional coverage deviation, often residing on the empirical Pareto frontier of the trade-off. Depending on coverage, SLS either matches or outperforms conformal, latent space, and density-based pipelines, particularly in cases with complex conditional structures.

Figure 3: On real-world regression datasets, SLS regions are more compact and achieve accuracy closer to target coverage, particularly at lower deviation values.

Theoretical and Practical Implications

The formal analysis clarifies that SLS regression achieves two major objectives not simultaneously addressed by previous approaches:

1. Direct targeting of conditional coverage and minimal volume: Unlike CDE+plugin or marginally valid conformal procedures, SLS bridges the gap, supporting direct, per-covariate high-density region estimation.
2. Expressivity versus computational tractability: By leveraging volume-preserving flows and their unions, SLS manages to encode highly expressive and covariate-adaptive set boundaries without resorting to sampling or post-hoc calibration, with scalable analytic or proxy objectives.

These capabilities suggest that SLS regression can serve as a robust foundation for rigorous uncertainty quantification in scenarios ranging from high-dimensional regression to simulation-based inference, especially where classical parametric or kernel density models are inapplicable.

Future Perspectives

Potential directions for advancing SLS include incorporating more structured or domain-informed frontier parameterizations (e.g., incorporating physical constraints or graph-structured data priors), extensions to discrete or mixed response types, and expanding the bi-level optimization to incorporate higher-moment coverage criteria.

Given the modular architecture of the framework, integration with large foundation models for $\mathcal{A}(X)$0 or fully amortized quantile estimators is also a practical avenue, potentially scaling SLS regression to structured output prediction or nontrivial manifold domains.

Conclusion

Super-Level-Set regression reframes predictive set estimation by direct geometric optimization of region boundaries at fixed conditional coverage. The introduction of a differentiable, shrinking-window objective, flexible volume-preserving frontier functions, and an alternating estimation scheme yields a robust, theoretically sound, and practically effective approach to minimal-volume high-density region computation. Empirical results underline the method's adaptivity, efficiency, and robustness, particularly in challenging or misspecified settings. SLS provides a template for further work at the intersection of geometric learning, distributional regression, and uncertainty quantification in high-dimensional statistical learning.