Deep Single-Index Fréchet Regression

Abstract: Predicting outputs that are located in non-Euclidean spaces, such as probability distributions, networks, and symmetric positive-definite matrices, is becoming increasingly important in modern data analysis, particularly when inputs are high-dimensional. We propose DeSI (Deep Single-Index Fréchet Regression), a semiparametric framework for regression with metric space-valued outputs and multivariate inputs that assumes a single-index structure for the conditional Fréchet mean. DeSI estimates an interpretable index direction, which quantifies the relative importance of inputs, using a deep neural network, and performs Fréchet regression along the resulting one-dimensional index in the target metric space. This structure mitigates the curse of dimensionality while retaining interpretability, which stands in contrast to standard deep neural networks. We establish theoretical guarantees for DeSI, including uniform approximation and convergence rates, and demonstrate its strong predictive performance through simulations on distributions, networks, and symmetric positive-definite matrices, as well as an application to compositional mood data from New Jersey.

• The paper introduces a novel framework that integrates a deep neural network for index learning with local Fréchet regression to predict metric space-valued outcomes.
• It demonstrates superior performance and stability across simulations with SPD matrices, networks, and distributions by achieving lower prediction errors.
• The approach offers interpretable insights through a learned single-index direction, enhancing regression of complex data and enabling rigorous real-world applications.

Deep Single-Index Fréchet Regression: Semiparametric Modeling with Metric Space-Valued Outputs

Introduction and Context

Regression with non-Euclidean responses (random objects in metric spaces, e.g., distributions, networks, SPD matrices, compositions) is central in contemporary statistical machine learning. Classical methods are inadequate due to the absence of algebraic structure in the output space, prohibiting standard operations required by conventional regression frameworks. Existing approaches—such as projection into Euclidean subspaces [faraway2014regression], kernel methods [hein2009robust], and Fréchet regression [petersen2019frechet]—offer only partial solutions, with limitations in scalability, flexibility, or interpretability.

The Deep Single-Index Fréchet Regression (DeSI) framework (2606.06957) constitutes a significant methodological advancement: it leverages deep learning to estimate a flexible yet interpretable index that projects high-dimensional predictors to a single scalar, while performing nonparametric regression (local Fréchet regression, LFR) along this intrinsic dimension. This structure directly targets the curse of dimensionality, enables model interpretability (via the learned direction in covariate space), and is supported by sharp theoretical guarantees.

Methodological Framework

Model Architecture

The DeSI framework decomposes prediction into the following sequence: multivariate input $X \in \mathbb{R}^p$ is projected to a scalar latent index via a deep neural network, which produces input-dependent or global direction vectors. The output $Y$ resides in a general metric space $(\Omega, d)$. The predicted Fréchet mean conditional on $X$ is assumed to follow a single-index structure:

$m(X) = \mathbb{E}_{\oplus}(Y|X) = \zeta(g_0(X)),$

where $g_0(X)$ is the learned single-index mapping (possibly nonlinear/nonlinear in $X$) and $\zeta: \mathbb{R} \to \Omega$ is a link function learned nonparametrically via LFR.

Figure 1: Overview of the DeSI framework. A DNN maps $X$ to a one-dimensional index, followed by LFR along this axis in the output metric space.

This architecture combines two essential modules:

• DNN for index learning: A multilayer neural network with Leaky ReLU activations is used. The final layer outputs either a subject-specific direction $\widehat\theta(X)$ (normalized to unit norm) or an aggregated global estimator $Y$0, yielding index values $Y$1.
• Local Fréchet Regression (LFR): For each projected $Y$2, LFR finds the Fréchet mean of outputs $Y$3 weighted by kernel functions of the distance between $Y$4 and $Y$5, using a learnable bandwidth $Y$6.

The framework is optimized end-to-end under a normalized squared metric prediction loss plus regularization on bandwidth, with simultaneous updates of network parameters and bandwidth.

Theoretical Properties

Universal approximation results hold for the neural index: for any target single-index function, arbitrarily close uniform approximation is achieved by sufficiently wide and deep DNNs. Precise convergence rates are established under standard regularity conditions (kernel, identifiability, geometric assumptions on $Y$7):

• The estimation error for the index mapping propagates into the LFR stage. The final estimator attains the rate:

$Y$8

where $Y$9 is the index estimation error, and $(\Omega, d)$0 is the LFR bandwidth.

This rate adapts minimax-optimal LFR rates to settings where the projection index is learned, quantifying the imprecision induced by neural network estimation in the first stage.

Empirical Results

Simulation Studies

Three distinct metric spaces are considered:

• Symmetric Positive-Definite (SPD) matrices (log-Cholesky metric)
• Networks (graph Laplacians, Frobenius metric)
• Univariate probability distributions (Wasserstein metric)

  Figure 2: MPE across simulation settings (SPD matrices, networks, distributions; various link functions). Boxplots show predictive error distributions for DeSI and competitors. DeSI yields lower errors as $(\Omega, d)$1 increases and is uniformly best or nearly best across settings.

Simulation results establish that DeSI achieves uniformly lower mean prediction error (MPE) than global Fréchet regression (GFR), single-index linear Fréchet regression (IFR), and deep Fréchet regression (DFR), with the margin increasing in $(\Omega, d)$2 and output complexity. Standard deviations demonstrate the stability of the estimator.

Estimation error for the true index direction $(\Omega, d)$3 is also smallest for DeSI. This is particularly pronounced as model complexity (nonlinearity, network-valued output) increases, indicating DeSI’s capacity to recover latent predictor structure, even when a global linear index is suboptimal.

Figure 3: MPE for index direction estimation. Boxplots show accuracy in recovering $(\Omega, d)$4 for DeSI and IFR across output types. Lower values indicate superior index recovery, favoring DeSI especially in non-Euclidean and nonlinear scenarios.

Real Data Analysis: Compositional Mood Data

Application to compositional mood data (proportion of time spent in affective states, mapped to the unit simplex) from a large US labor survey demonstrates DeSI’s capacity for interpretable scientific inference.

Figure 4: Mood composition stratified by life satisfaction level. Higher life satisfaction sharply increases “very good” and “mildly pleasant” affect, with a corresponding drop in negative moods, captured by the dominant learned index direction.

DeSI highlights life satisfaction as the dominant driver among a rich covariate set—this interpretability is achieved by inspection of the index coefficients. DeSI achieves an MPE improvement of 6% over IFR and 34% over GFR, indicating substantial gains in real-world settings.

Implications and Future Directions

DeSI provides a principled middle ground between rigid linear structure and black-box deep learning: it restricts the predictor-to-response map to a single-index form but allows the index itself to be nonlinear and highly adaptive. This enables interpretability (via the learned direction vector), statistical tractability, and empirical superiority. The method is robust to modest misspecification of the single-index structure; when the true response is not strictly a function of a single index, DeSI still yields optimal or near-optimal predictive linear projections.

Potential extensions include:

• Multi-index models: Learning multiple nonlinear projections to capture more complex structures, with interpretability traded for improved expressiveness.
• Manifold-valued inputs: Extension to responses and predictors in non-Euclidean domains (e.g., joint modeling, functional data on manifolds).
• Scalable computation: GPU implementation and hybrid acceleration for very large datasets with complex metric geometries.

Conclusion

Deep Single-Index Fréchet Regression provides a semiparametric, interpretable, and theoretically justified solution to regression problems with high-dimensional predictors and metric space-valued outputs. The framework unifies the flexibility of deep learning with the statistical and inferential advantages of the single-index paradigm, with strong empirical performance in both simulated and real domains. This approach has broad relevance for scientific applications involving random objects, complex outputs, or structured data modalities.

Reference: "Deep Single-Index Fréchet Regression" (2606.06957)