---
title: Variable Basis Mapping (VBM)
url: https://www.emergentmind.com/topics/variable-basis-mapping-vbm
type: topic
---

# Variable Basis Mapping (VBM)

Variable Basis Mapping (VBM) refers to a penalized basis learning methodology for high-dimensional ordinal classification problems. The VBM framework, as developed in Kim et al. (2024), extends sparse multiclass linear discriminant analysis (LDA) by introducing per-variable ordinal weights and a weighted group-lasso penalty, thereby enabling the selection of variables that exhibit both discriminative and order-concordant behavior with respect to the ordinal response. VBM is designed for regimes with high-dimensional feature spaces (\(p \gg N\)), where interpretability and variable selection are critical.

## 1. Formulation of the Ordinal-Weighted Sparse Basis Learning Problem

Let \(X \in \mathbb{R}^p\) denote a feature vector and \(y \in \{1,\ldots,K\}\) the ordinal class label. The VBM method assumes a common-covariance Gaussian model:
\[
X \mid (y = g) \sim N(\mu_g, \Sigma), \quad g = 1, \dots, K.
\]
Standard multiclass LDA seeks a \((K-1)\)-dimensional basis \(Z \in \mathbb{R}^{p \times (K-1)}\) that maximizes separation between groups. The unpenalized estimator is:
\[
\Psi = \Sigma^{-1} M = \arg\min_{Z \in \mathbb{R}^{p \times (K-1)}} \operatorname{tr}\left(\tfrac{1}{2} Z^T \Sigma Z - Z^T M\right),
\]
where \(\Sigma\) is the (pooled) within-group covariance and \(M\) determines the between-group means.

To promote sparsity and preferentially select order-concordant variables, VBM introduces the following penalized objective with per-variable ordinal weights \(w_j\):
\[
\widehat Z^{\mathrm{ord}}_{\eta,\lambda}
= \arg\min_{Z \in \mathbb{R}^{p \times (K-1)}}
\operatorname{tr}\left(\tfrac{1}{2} Z^T \widehat\Sigma Z - Z^T \widehat M\right)
+ \sum_{j=1}^p \lambda \eta^{1-w_j} \|Z_{j,:}\|_2,
\]
where \(\lambda > 0\) sets overall sparsity and \(\eta \ge 1\) amplifies penalization for variables with smaller \(w_j\) [2208.10690].

## 2. Construction of Ordinal Weights via Two-Step Kendall’s Tau Procedure

Variable selection is guided by ordinal weights \(w_j \in [0,1]\), constructed via a two-step process leveraging Kendall’s tau statistics:

- **Global Kendall’s Tau (\(\hat{\tau}_j\))**: Measures correlation between \(X_{ij}\) and \(y_i\) across all samples.
  \[
  \hat{\tau}_j = \frac{2}{N(N-1)} \sum_{1 \leq i < k \leq N} \operatorname{sgn}(X_{kj} - X_{ij}) \operatorname{sgn}(y_k - y_i)
  \]
- **Group-Mean Kendall’s Tau (\(\tilde{\tau}_j\))**: Assesses monotonicity of group means.
  \[
  \tilde{\tau}_j = \frac{2}{K(K-1)} \sum_{1 \leq g < h \leq K} \operatorname{sgn}(\hat{\mu}_j^{(h)} - \hat{\mu}_j^{(g)})
  \]
  
For thresholds \(0 < \theta_1, \theta_2 < 1\), variable \(j\) is assigned
\[
w_j = \begin{cases}
1, & \text{if } |\hat{\tau}_j| > \theta_1 \text{ and } |\tilde{\tau}_j| > 1-\theta_2 \\
0, & \text{otherwise}
\end{cases}
\]
Step 1 eliminates “noise” variables; Step 2 detects variables whose class means are strictly monotone. Theorems guarantee that this rule selects true order-concordant variables with high probability under mild assumptions [2208.10690].

## 3. Optimization via Block-Coordinate Descent

The minimization of the VBM objective is efficiently solved by block-coordinate descent, exploiting the group-lasso structure. For each row \(j\) in \(Z\), the algorithm updates as follows:
1. Compute the partial-residual vector:
   \[
   a_j = \widehat M_{j,:} - \sum_{k \neq j} \widehat\Sigma_{jk} Z^{(t)}_{k,:}
   \]
2. Set \(\lambda_j = \lambda \eta^{1-w_j}\) and \(\sigma_{jj} = \widehat\Sigma_{jj}\).
3. Update row \(j\):
   \[
   Z^{(t+1)}_{j,:} = \frac{1}{\sigma_{jj}} \left(1 - \frac{\lambda_j}{\|a_j\|_2}\right)_+ a_j
   \]
The procedure converges to the global optimum due to the convexity and block-separability of the objective.

## 4. Theoretical Guarantees in High-Dimensional Regimes

VBM exhibits non-asymptotic oracle properties under high-dimensional scaling. Key sets include:

- Discriminant variables \(J_{\text{disc}} = \{j : \Psi_{j,:} \neq 0\}\)
- Ordinal variables \(J_{\text{ord}} = \{j : \mu_j^1 \leq \cdots \leq \mu_j^K\}\)
- Ordinal-discriminant \(J^{\text{ord}}_{\text{disc}} = J_{\text{disc}} \cap J_{\text{ord}}\)

Selection consistency is achieved when tuning parameters \((\lambda, \eta)\) are chosen appropriately:
- For moderate \(\eta\), all discriminative variables are selected.
- For large \(\eta\), only variables that are both discriminative and order-concordant are selected.

Estimation bounds (in \(\ell_{\infty,2}\)) are provided:
\[
\|\widehat Z^{\mathrm{ord}}_{\eta,\lambda} - \Psi\|_{\infty,2} = O(\phi \eta \lambda)
\]
in probability, where \(\phi\) is a compatibility constant. High-dimensional consistency requires \(\log(pd) d^2 / N \rightarrow 0\), \(\lambda \to 0\) sufficiently slowly, and \(\eta\lambda \to \infty\) [2208.10690].

## 5. Post-Screening and Data-Adaptive Refinement

In practical applications, data-adaptive thresholding is deployed. The two-step weights can be refined by:
- Initial screening using ANOVA F-tests to screen noise from variables with nontrivial mean differences.
- Adaptive selection of \(\theta_1\) and \(\theta_2\) based on empirical distributions of \(\hat{\tau}_j\).
- This adaptive procedure maintains strict separation between order-concordant and non-monotone variables.

## 6. Interpretability and Sparsity of the Learned Representation

The group-lasso penalty in VBM leads to row-sparsity in the learned basis: only a small subset of variables contributes to the \((K-1)\)-dimensional discriminant subspace. Variables with monotonic class means under \(y\) incur less regularization (\(\lambda_j = \lambda\)) and are preferentially retained when \(\eta > 1\), while non-monotone or noisy features are heavily penalized and typically excluded.

Each selected variable corresponds to a row in \(Z\) and can be directly mapped to interpretable patterns of monotone group-mean shifts in the projected subspace. This facilitates intelligible variable selection, particularly useful in domains such as genomics, where the interpretability of the selected genes is paramount.

Practical results include:
- In low-dimensional synthetic settings, VBM recovers the true set \(J^{\text{ord}}_{\text{disc}}\) under suitable \(\eta\).
- In large-scale gene expression datasets, VBM selects a highly sparse subset (typically 7–20 out of >10,000 genes) while maintaining competitive or superior classification error rates compared to nominal LDA or ordinal logistic regression [2208.10690].

Source: https://www.emergentmind.com/topics/variable-basis-mapping-vbm