---
title: 'MVPP: Multi-View Predictive Partitioning'
url: https://www.emergentmind.com/topics/multi-view-predictive-partitioning-mvpp
type: topic
---

# MVPP: Multi-View Predictive Partitioning

Multi-View Predictive Partitioning (MVPP) is a clustering methodology for high-dimensional data in which each observation is represented by paired multivariate vectorial “views.” It is designed to discover clusters that exhibit strong within-cluster predictive agreement between these paired representations, rather than simple geometric proximity. MVPP utilizes the two-block partial least squares (TB-PLS) regression model within each cluster, with a novel objective based on predictive influence—a measure derived from an analytic leave-one-out cross-validation (LOOCV) statistic. As a result, the algorithm directly partitions data such that predictive relationships between views are maximized within clusters, enabling the identification of structure ignored by methods that rely solely on geometric similarity [1202.0825].

## 1. Two-Block Partial Least Squares in Cluster Modeling

MVPP models the dependency structure between paired high-dimensional views $\mathbf x_i \in \mathbb R^p$ and $\mathbf y_i \in \mathbb R^q$ observed for each data point $i$. Within each cluster of $n$ i.i.d. observations, TB-PLS introduces $R$ latent factors, decomposing the data as
$$
\mathbf X = \sum_{r=1}^R \mathbf t^{(r)}\mathbf p^{(r)T} + \mathbf E_x, \quad
\mathbf Y = \sum_{r=1}^R \mathbf s^{(r)}\mathbf q^{(r)T} + \mathbf E_y,
$$
where $\mathbf t^{(r)} = \mathbf X\,\mathbf u^{(r)}$, $\mathbf s^{(r)} = \mathbf Y\,\mathbf v^{(r)}$, and each $(\mathbf t^{(r)}, \mathbf s^{(r)})$ pair maximizes covariance. The optimal directions are obtained via singular value decomposition (SVD) of $\mathbf X^\top\mathbf Y = \mathbf U\,\boldsymbol\Lambda\,\mathbf V^\top$, identifying $\mathbf u^{(r)}, \mathbf v^{(r)}$ as the $r$th singular vectors. Final regression of $\mathbf Y$ on $\mathbf X$ proceeds via
$$
\mathbf Y \approx \mathbf X\boldsymbol\beta, \quad \boldsymbol\beta = \sum_{r=1}^R \mathbf u^{(r)}\,g^{(r)}\,\mathbf q^{(r)T},
$$
with $g^{(r)}$ extracted from the inner regression.

The TB-PLS framework performs joint dimensionality reduction and predictive modeling, which is advantageous in high-dimensional settings frequently encountered in domains such as web mining and genomics.

## 2. Predictive Influence and the PRESS-Derived Objective

MVPP replaces geometric similarity with a predictive influence criterion rooted in a cluster-wise LOOCV PRESS statistic:
$$
J = \frac{1}{n}\sum_{i=1}^n \|\mathbf y_i - \mathbf x_i\boldsymbol\beta_{(-i)}\|^2.
$$
A closed-form approximation avoids costly leave-one-out retraining:
$$
J \approx \frac{1}{n}\sum_{i=1}^n \Bigg\|\, \mathbf e_i - t_i^2\,\mathbf E_{y,i} - \mathbf b_i \,\Bigg\|^2\,/\,\Big[(1 - t_i^2)(1 - s_i^2)\Big]^2,
$$
with $\mathbf e_i = \mathbf y_i - \mathbf x_i\boldsymbol\beta$, $t_i$, $s_i$ the projected scores, $\mathbf E_{y,i}$ the row residuals in $\mathbf Y$, $\mathbf b_i=h_i s_i \mathbf y_i$, and $h_i=s_i - g t_i$.

The predictive influence of each point $i$ is given by the total derivative:
$$
\boldsymbol\pi_i = \frac{\partial J}{\partial(\mathbf x_i, \mathbf y_i)} \in \mathbb{R}^{p+q}
$$
where $\|\boldsymbol\pi_i\|^2$ quantifies how much $J$ would change if $(\mathbf x_i, \mathbf y_i)$ were perturbed. Points with high influence are poorly predicted by the TB-PLS model; this property anchors the MVPP clustering objective.

## 3. MVPP Algorithm and Optimization

The MVPP algorithm simultaneously estimates cluster assignments and cluster-specific TB-PLS models by minimizing the sum of predictive influences within each cluster. The main steps are:

1. Randomly assign each data point to one of $K$ clusters (initial $\{\mathcal C_k\}$).
2. **E–step**: For each cluster $k$, fit a TB-PLS model with $R$ factors and compute $\boldsymbol\pi_i^{(k)}$ for all $i \in \mathcal C_k$.
3. **P–step**: For each $i$, re-assign to cluster $k$ that minimizes $\|\boldsymbol\pi_i^{(k)}\|^2$.
4. Iterate E– and P–steps until assignments stabilize or a maximum number of iterations is reached.

Each E–step reduces within-cluster influence, and each P–step assigns each point to its minimal-influence cluster, which guarantees convergence to a local minimum.

## 4. Model Selection and Computational Complexity

Selecting the number of clusters $K$ is performed by running MVPP for $K=1,\dots,K_{\max}$ and choosing $K$ that minimizes the average PRESS across clusters, which typically identifies the ground-truth $K$, as PRESS penalizes overfitting. The number of PLS latent components $R$ per cluster can similarly be determined by PRESS minimization, though a default of $R=1$ per cluster is often sufficient unless further reduction in PRESS is achieved by more components.

The algorithm’s per-iteration complexity comprises:
- $K$ TB-PLS fits, each requiring SVD of a $p \times q$ matrix: $O(p^2q + pq^2)$ operations per fit.
- For each of $n$ points and $K$ clusters, computing and comparing influence norms: $O(nK(p+q))$.
Total per-iteration: $O\left(K(p^2q + pq^2) + nK(p+q)\right)$. For $p, q \gg n$, the SVD cost dominates; the analytic PRESS/influence calculation circumvents the need for $n$ separate fits.

## 5. Empirical Performance and Comparison to Other Methods

MVPP was assessed on both synthetic and real-world high-dimensional data with paired-view structure. In synthetic “Scenario A” (geometric clusters), MVPP demonstrated robustness to noise and superior clustering accuracy and LOOCV prediction error relative to alternatives such as WCC, MV-CCA, and multi-view kernel approaches. In “Scenario B” (confounding geometry), where clusters are geometrically intermixed but have distinct predictive maps, conventional geometric methods failed to recover clusters, whereas MVPP succeeded.

On web-mining datasets (WebKB-2, WebKB-4) and the Citeseer citation network—a setting with dual high-dimensional views (e.g., text and link features)—MVPP reached approximately 100% (WebKB-2) and 80% (WebKB-4) clustering accuracy, outperforming geometric baselines (which scored 50–70%), and consistently provided the lowest LOOCV mean squared error.

## 6. Algorithmic Distinctions and Practical Significance

MVPP is distinct in its choice of within-cluster predictive modeling as the basis for clustering, as opposed to classical Euclidean or geometric proximity. By leveraging TB-PLS, it enables explicit modeling of inter-view relationships even in high-dimensional settings, where geometric distances are less informative or dominated by noise. The focus on minimizing predictive influence through an analytic PRESS extension allows for efficient evaluation of point-wise cluster fit, fostering the discovery of clusterings that are invisible to methods relying solely on geometric structure. This framework is especially important in application areas such as web mining and genomics, where multiview, high-dimensional data are common and predictive relationships underpin the relevant structure [1202.0825].

Source: https://www.emergentmind.com/topics/multi-view-predictive-partitioning-mvpp