Graph-based Semi-Supervised Learning via Maximum Discrimination

Abstract: Semi-supervised learning (SSL) addresses the critical challenge of training accurate models when labeled data is scarce but unlabeled data is abundant. Graph-based SSL (GSSL) has emerged as a popular framework that captures data structure through graph representations. Classic graph SSL methods, such as Label Propagation and Label Spreading, aim to compute low-dimensional representations where points with the same labels are close in representation space. Although often effective, these methods can be suboptimal on data with complex label distributions. In our work, we develop AUC-spec, a graph approach that computes a low-dimensional representation that maximizes class separation. We compute this representation by optimizing the Area Under the ROC Curve (AUC) as estimated via the labeled points. We provide a detailed analysis of our approach under a product-of-manifold model, and show that the required number of labeled points for AUC-spec is polynomial in the model parameters. Empirically, we show that AUC-spec balances class separation with graph smoothness. It demonstrates competitive results on synthetic and real-world datasets while maintaining computational efficiency comparable to the field's classic and state-of-the-art methods.

• The paper introduces AUC-spec, a graph-based SSL algorithm that optimizes the AUC to enhance class discrimination over graph-induced representations.
• It employs a continuous AUC approximation with power iteration dynamics to balance graph smoothness and discriminative updates, extending naturally to multi-class scenarios.
• Empirical and theoretical results demonstrate improved classification performance, statistical efficiency, and computational advantages over classical smoothness-based SSL methods.

Graph-Based Semi-Supervised Learning via Maximum Discrimination: AUC-spec

Overview

The paper "Graph-based Semi-Supervised Learning via Maximum Discrimination" (2602.08042) introduces AUC-spec, a graph-based semi-supervised learning (SSL) algorithm that optimizes the Area Under the ROC Curve (AUC) to maximize class separation in graph-induced representations. The principal motivation is to address the intrinsic limitations of classical graph-based SSL approaches, which prioritize smoothness and label consistency, often at the expense of discriminative power in scenarios with complex label structures or overlapping manifolds.

Limitations of Classical Graph-Based SSL

Traditional graph-based SSL methods—including Label Propagation (LP), Label Spreading, and leading eigenvector-based techniques—are predicated on enforcing label agreement and smoothness over the data manifold. These approaches are effective under strong cluster assumptions, where label boundaries align with graph connectivity patterns. However, they exhibit substantial limitations:

• Spiked/Uninformative Solutions: As the proportion of unlabeled data increases, solutions often become degenerate, matching labels at labeled points but losing meaningful variation elsewhere in the graph.
• Spectral Misalignment: Discriminative information may not reside in leading Laplacian eigenvectors, especially in datasets where label structures are not congruent with primary modes of variation in the data graph.
• Rigidity: Enforcing label values at labeled nodes can be overly restrictive and yield suboptimal, non-discriminative solutions in the presence of partially overlapping clusters or latent continuous labels.

The failure modes are empirically illustrated on synthetic datasets with structured overlap and on real-world benchmarks, demonstrating the shortfall of classical methods in producing maximally discriminative representations.

The AUC-spec Formulation

AUC-spec fundamentally reorients the graph-based SSL objective: rather than enforcing label agreement, it seeks to explicitly maximize the separability between labeled points of distinct classes under the graph-induced representation. The core of the method is the following joint optimization:

$$J(\mathbf{v}) = \mathbf{v}^T L \mathbf{v} - \gamma \, \mathrm{AUC}(y_c, \mathbf{v}_c)$$

where $\mathbf{v}$ is the predictive vector for all nodes, $L$ is the graph Laplacian, $y_c$ and $\mathbf{v}_c$ denote the observed labels and corresponding predictive values, and $\gamma$ controls the tradeoff between graph smoothness and AUC-driven discrimination.

Key technical innovations include:

• Continuous AUC Approximation: Replacing the non-differentiable indicator in AUC with a smooth sigmoid-based surrogate to facilitate efficient gradient-based optimization.
• Power Iteration Dynamics: Integrating the AUC-driven gradient into power iteration updates over the random walk Laplacian, so the representation evolves by simultaneously smoothing and maximizing inter-class separation.
• Multi-Class Extension: A natural generalization via one-vs-rest optimization of class-specific AUC objectives, supporting multi-class SSL.

The computational complexity is competitive with state-of-the-art, scaling efficiently with the number of labeled points and leveraging sparsity in the underlying graph.

Theoretical Analysis under Manifold and Mixture Models

The paper provides a detailed theoretical study in the context of mixture models and product-manifold settings:

• Label Complexity: For product-of-manifolds models, the number of labeled points needed for non-trivial discrimination by AUC-spec scales polynomially with model parameters and logarithmically with the total number of points, in contrast with the poor asymptotics for LP.
• Spectral Localization: Theoretical guarantees show that, under mild assumptions, the optimal solution aligns with discriminative directions in the spectrum, not necessarily the leading eigenvectors, and retains smoothness with respect to the data manifold.
• AUC-optimal Vectors: The solution vector exhibits monotonicity or strong alignment with latent discriminative factors, as opposed to the spiked, unsmooth outcomes of Laplacian-based label enforcement in the infinite-sample regime.

These results collectively demonstrate that AUC-spec overcomes key theoretical bottlenecks of existing graph-based SSL techniques.

Empirical Results

Comprehensive empirical evaluation is conducted across a diversity of domains: structured synthetic data (Ring-of-Gaussians), image classification benchmarks (MNIST, Fashion MNIST, CIFAR-10), and tabular data (MAGIC Gamma Telescope). The main findings are:

• Superior Discriminative Power: AUC-spec reliably achieves higher or comparable AUC and accuracy across all datasets, with particularly strong gains in low-label regimes and for more challenging manifolds where conventional methods fail to find discriminative directions.
• Statistical Efficiency: Relatively few labeled points (on the order of tens) suffice to achieve high AUC on nontrivial graph structures, matching theoretical predictions.
• Computational Advantages: The method demonstrates competitive or lower runtime than state-of-the-art LP, p-Laplacian, and Poisson-based algorithms, especially as the number of labels decreases—a critical attribute for scalable SSL.

Notably, AUC-spec exhibits robustness to the choice of threshold on predictive vectors, with potential for further gains via more principled calibration.

Implications and Future Directions

AUC-spec reframes graph-based SSL as a problem of maximizing empirical class separability rather than pure label regularity, introducing a new regime of discrimination-aware graph-based learning. The approach is particularly well-suited for:

• Settings with overlapping or complex label boundaries, where classical smoothness-based propagation is inadequate.
• Scenarios with abundant unlabeled data and few labels, due to favorable sample complexity scaling and computational tractability.

Potential avenues for further research include:

• Integrating AUC-spec with higher-order graph regularization or robust graph smoothing techniques (e.g., p-Laplacian, Poisson-based methods).
• Adaptive trade-off strategies to dynamically balance smoothness with discrimination during training.
• Extension to partial-AUC, fairness-aware, and other advanced discrimination criteria.
• Scalable algorithms for massive graphs, leveraging distributed or low-rank graph approximations.

Conclusion

The AUC-spec algorithm establishes a new perspective for semi-supervised learning on graphs, explicitly targeting maximal discrimination via AUC optimization rather than rigid label consistency. Both theoretical and empirical results validate its advantages over established methods, especially in regimes characterized by complex label structures and limited labeled data. The framework provides a versatile foundation for future developments in discrimination-aware graph-based SSL.

(2602.08042)