Unified Linear Comparative Analysis (ULCA)

• ULCA is a unified framework that generalizes PCA, LDA, and cPCA into a single convex optimization scheme for comparative analysis.
• It employs tunable parameters to separately control within-class emphasis, between-class separation, and background suppression.
• The method solves a trace-ratio optimization problem to yield interpretable, real-time projections, with extensions available for higher-order tensor data.

Unified Linear Comparative Analysis (ULCA) is a unified linear dimension-reduction and comparative-analysis methodology that generalizes and interpolates core objectives of discriminant analysis, contrastive learning, and principal component analysis in a rigorous algebraic framework. ULCA provides a single convex optimization scheme with a tunable parametrization, supporting both discriminative and contrastive axes for exploratory analysis, group comparison, and visualization in high-dimensional settings. The ULCA framework allows analysts to flexibly control within-class emphasis, between-class separation, and background suppression, yielding a versatile toolkit that subsumes PCA, LDA, and cPCA as special cases. Extension to the multilinear (tensor) setting is provided via TULCA, which enables comparative analysis across higher-order structures.

1. Unifying Comparative Linear Methods

ULCA was developed to address limitations of traditional linear methods used in comparative analysis of high-dimensional data. Classical dimensionality-reduction techniques typically focus on a singular analytic target: PCA maximizes overall variance, LDA maximizes between-class separation relative to within-class spread, and cPCA identifies target-specific variance directions that are suppressed in a background group. However, these approaches offer limited flexibility when the analytic context requires both discriminant and contrastive axes—such as when comparing multiple classes while suppressing known nuisance factors.

ULCA constructs a single trace-ratio (Rayleigh quotient) objective that jointly maximizes between-class scatter, modulates within-class variance for selected groups, and can contrast out or suppress variance due to background groups. Analysts control three sets of nonnegative weights, each governing the emphasis on target (within-class variance maximized), background (within-class variance suppressed), and between-class (discriminant) objectives for each group. This parameterization enables continuous interpolation between the special cases:

• Setting only target weights (pure PCA)
• Activating only between-class weights (LDA)
• Modulating target and background weights (cPCA)
• Arbitrary mixing for complex comparative tasks (Okami et al., 26 Jul 2025, Fujiwara et al., 2021)

2. Mathematical Formulation and Optimization

Consider a dataset $X \in \mathbb{R}^{n \times d}$ and class labels $y \in \{1,\dots,c\}^n$. Denote $\Sigma_j$ as the within-class covariance for class $j$ and $B_j$ as the between-class covariance contribution. ULCA introduces three sets of weights:

• $w_j^+$: amplification of within-class variance for class $j$
• $w_j^-$: suppression (background subtraction) of within-class variance for class $j$
• $w_j^b$: amplification of between-class variance for class $y \in \{1,\dots,c\}^n$0

With small regularizers $y \in \{1,\dots,c\}^n$1, define

$y \in \{1,\dots,c\}^n$2

$y \in \{1,\dots,c\}^n$3

ULCA seeks a projection matrix $y \in \{1,\dots,c\}^n$4 (with $y \in \{1,\dots,c\}^n$5) maximizing

$y \in \{1,\dots,c\}^n$6

This trace-ratio problem is solved via either the Dinkelbach algorithm—iteratively updating $y \in \{1,\dots,c\}^n$7 and computing top generalized eigenvectors of $y \in \{1,\dots,c\}^n$8—or manifold optimization (e.g., Riemannian trust-region methods on the Stiefel/Grassmann manifold). The ULCA solution yields an embedding $y \in \{1,\dots,c\}^n$9 optimized for the analyst's chosen comparative emphasis (Okami et al., 26 Jul 2025, Fujiwara et al., 2021).

3. Special Cases and Theoretical Properties

ULCA's parameterization admits the following classical methods as special cases:

Objective	$\Sigma_j$0	$\Sigma_j$1	$\Sigma_j$2	$\Sigma_j$3	$\Sigma_j$4, $\Sigma_j$5
PCA	All 1	All 0	All 0	0	All-data covariance, $\Sigma_j$6
LDA	All 1	All 0	All 1	0	Between, Within
cPCA	Target 1	Others 1	All 0	0	Target minus background

ULCA is solved via a single convex-ratio problem and always produces orthonormal projections. With standardization, results are invariant to feature scaling. Analytical control of regularization ensures positive-definiteness in high-dimension, low-sample scenarios. The generalized eigenproblem produces a closed-form solution; computational complexity is dominated by covariance computation $\Sigma_j$7 and eigen-decomposition $\Sigma_j$8 (Okami et al., 26 Jul 2025, Fujiwara et al., 2021).

4. Algorithmic Workflow and Interactive Refinement

The standard ULCA pipeline consists of:

1. Centering: Compute overall and group means; center as needed.
2. Covariance computation: Form within- and between-class covariance blocks.
3. Weight application: Aggregate $\Sigma_j$9, $j$0 using the analyst's weight choices.
4. Eigenproblem solution: Solve for the top $j$1 orthonormal directions.
5. Embedding: Project data via $j$2.

Interactive ULCA systems augment this with direct manipulation: analysts adjust $j$3 via sliders or graphical feedback (e.g., dragging ellipses representing group centroids or variances in 2D projections). The system re-solves the ULCA optimization in real time using backward parameter selection (nonlinear least-squares over the weight vectors), preserving immediate interpretability and iterative exploration (Fujiwara et al., 2021).

A typical pseudocode for eigen-decomposition with Dinkelbach iteration:

$B_j$0

5. Relation to UCA, cPCA, and Other Comparative DR Methods

Unlike classical cPCA, which requires manual contrast parameter tuning (often via grid search), ULCA absorbs such balancing into its weight vectors, which can be set analytically, interactively, or heuristically. Unique Component Analysis (UCA) further extends cPCA by providing a tuning-free, convex optimization that supports multi-background contrast via constraints or dual optimization, but does not natively integrate discriminant analysis objectives within a single convex ratio; ULCA directly combines discriminant and contrastive elements (Tu et al., 2021).

ULCA's trace-ratio construction generalizes both the Rayleigh quotients for discriminative and variance-maximizing projections and the trace-difference (Dinkelbach) forms underlying classical and contrastive objectives. The flexibility of parameterization allows superior analyst control relative to pooled background or pairwise subtraction methods in cPCA/UCA, while retaining closed-form and interpretable embeddings (Fujiwara et al., 2021, Okami et al., 26 Jul 2025).

6. Multilinear Generalization: TULCA

The tensor extension, TULCA (Tensor Unified Linear Comparative Analysis), extends ULCA to N-way arrays by performing Rayleigh-quotient optimization independently along each mode (2 through N) after unfolding the tensor:

• For each mode $j$4, form mode-$j$5 unfoldings and within-/between-group covariance blocks.
• Solve $j$6 for orthonormal $j$7.
• Project core tensor $j$8.

TULCA preserves multilinear structures lost in flattening/matricization, facilitating comparative analysis across complex data (e.g., time-subject-sensor tensors). Empirically, TULCA outperforms matrix-based ULCA in settings where $j$9 or group structure varies along multiple tensor axes (Okami et al., 26 Jul 2025).

7. Applications, Strengths, and Limitations

ULCA and TULCA have demonstrated utility in:

• Comparative genomics and gene-expression analysis
• Differentiation of medical imaging modalities
• Network anomaly or baseline–difference detection
• Temporal or multi-period system log analysis (e.g., supercomputers, as in TULCA case studies)
• Mobile-health activity recognition and similar tasks requiring simultaneous discriminant and contrastive analysis

Key advantages include unified parameterization, built-in regularization, efficient eigen-solvability, and direct interpretability. Limitations include potential unintuitiveness in weight setting for large class numbers, possible overfitting in extremely high-dimensional low-sample contexts, and the inability of linear projections to capture nonlinear group separations. For tensor data, TULCA mitigates overfitting and structure loss innate to matricization approaches. Improper parameterization may produce spurious clusters, a caveat shared with all DR methodologies (Fujiwara et al., 2021, Okami et al., 26 Jul 2025).

---

In summary, Unified Linear Comparative Analysis provides a principled, highly flexible framework integrating the main linear comparative paradigms within a single mathematically tractable objective, with practical extensions to the tensor setting for advanced analytics in high-dimensional, structured data environments.