Fast Rates for Semi-Supervised Learning via Data-Augmentation Graph Regularization

Abstract: Self-supervised learning matches supervised accuracy from a fraction of the labels, but the labeled-sample efficiency behind this has lacked a theoretical explanation. We provide one. Data augmentation induces a similarity graph on the unlabeled data, so downstream learning on that graph is graph-Laplacian-regularized learning. We prove a fast transductive rate, $O(1/n_L)$ in the number of labels, in place of the supervised $O(1/\sqrt{n_L})$, by carrying the leave-one-out stability apparatus of Johnson and Zhang (JMLR 2007) over to the augmentation graph, and without the unrealistic assumptions of limit-based analyses (exact kernel, generalizing features). The bound makes augmentation quality explicit: the expected error is at most $C/n_L + R_{\mathrm{DA}}(y)$, where the data-augmentation alignment error $R_{\mathrm{DA}}(y)$ is the graph-cut mass of augmentations that cross a label boundary, so good augmentations let few labels suffice. The analysis uses a streamlined loss that drops the projector, negative-sample, and orthogonality overhead of standard objectives yet still recovers the top-$K$ ideal features in the infinite-data limit, the augmentation-kernel eigenspace studied by Zhai et al. The result explains the observed accuracy-versus-label-count curve rather than only bounding a generalization gap.

• The paper demonstrates a theoretical framework achieving fast O(1/n_L) rates via data-augmentation graph regularization in semi-supervised learning.
• It formalizes an augmentation graph using Laplacian regularization to enforce smooth class scores and promote label consistency.
• Experimental results show strong label efficiency, with competitive accuracy achieved using only 4% of available labels.

Fast Rates for Semi-Supervised Learning via Data-Augmentation Graph Regularization

Problem Statement and Motivation

The manuscript develops a formal explanation for the empirical label-efficiency phenomenon observed in self-supervised learning frameworks such as SimCLR and CPCv2, where comparable supervised accuracy is achieved using substantially fewer labels. Previous theoretical accounts characterized the infinite-data regime, showing that representation learning with data augmentations recovers leading eigenfunctions of an augmentation-induced similarity kernel. However, finite-label sample complexity and downstream accuracy-versus-label-count behavior—central to practical semi-supervised learning (SSL)—remained unquantified in existing theory.

Augmentation Graph: Formalization and Regularization

Data augmentations induce a probabilistic similarity graph on the unlabeled dataset, implemented via an edge-weighted kernel $k^{\mathrm{DAF}}$ defined by the expected co-occurrence of augmented views from shared seeds. The unlabeled sample forms the node set, with edge weights determined directly by augmentation statistics. Regularization is performed through a quadratic penalty involving the graph Laplacian (degree-normalized), enforcing smoothness of class scores across nodes connected by augmentation, thereby promoting feature invariance under augmentation.

The SSL objective is a convex combination of supervised loss on revealed labeled nodes and a Laplacian-plus-ridge regularization. Notably, the regularizer is independent of the labels and class score function $g$, resulting in a streamlined objective that omits collapse-prevention constraints (orthogonalization, projector dimension, negative samples) required in typical contrastive/non-contrastive losses.

Main Rate Results: Algorithmic Stability

Transductive Fast Rate via Leave-One-Out Stability

The core theoretical contribution is adapting the leave-one-out uniform stability framework of Johnson and Zhang [ZhangLaplacian] to the SSL setting defined by the augmentation graph. The key result is a transductive oracle inequality bounding the expected error at the unlabeled nodes by

$$\mathbb{E}_{S^L}\left[L_{\mathrm{loo}}\right] \leq \min_{g}\left\{\frac{1}{m}\sum_{i=1}^{m}\ell(g,z_i) + \lambda Q(g, S^U)\right\} + \frac{\sigma_\ell^2}{2 \lambda n_L} \cdot \frac{1}{m}\sum_{i=1}^{m} \mathbf{K}_{i,i}$$

where $n_L$ is the number of labeled nodes, $Q$ is the quadratic regularizer, and the stability premium is sharply $O(1/n_L)$—markedly faster than the standard supervised $O(1/\sqrt{n_L})$ rate.

Augmentation Quality: Data-Augmentation Alignment Error

The regularized oracle term is explicitly linked to a data-augmentation alignment error $R_{\mathrm{DA}}(y)$, which quantifies the normalized augmentation-graph cut mass between nodes of differing labels:

$$\mathbb{E}_{S^L}\left[\operatorname{err}\right] \leq \frac{C}{n_L} + R_{\mathrm{DA}}(y)$$

with $C$ (dependent on regularization, balanced components, and kernel trace) and $g$0 (zero iff augmentations are label-consistent). Empirically observed accuracy for SSL is explained: better label-consistent augmentations reduce $g$1, permitting strong performance with few labels.

Theoretical and Practical Implications

Characterizing Streamlined Objectives

The streamlined objective (omitting collapse penalties) is shown to recover the top-$g$2 eigenspace of the augmentation kernel in the large-data limit, matching the spectral characterization of various contrastive and non-contrastive self-supervised losses [haochen2021spectral, balestriero2022recover]. Rate-optimal feature learning is achieved without the overhead of projector dimension, negatives, or explicit orthogonality.

Conditions for Fast Rates

Component balance and bounded margin (separation of class score level sets) are critical for attaining the sharp $g$3 rate. Imbalanced components degrade the rate to the slower $g$4. The result is strictly transductive, addressing error propagation across the fixed unlabeled sample rather than generalization to unseen data; generalization bounds remain the slower $g$5 due to concentration inequalities.

Experimental Verification

Instrumentation with a planted-partition (stochastic block) graph validates the $g$6 stability mechanism and confirms additive versus multiplicative dependence on the augmentation graph cut. A CIFAR-10 study with frozen SimCLR features demonstrates label efficiency: supervised accuracy is achieved using only $g$7 of labels, with the downstream curve flattening well before all labels are used.

Relation to Prior Work

The analysis specializes, leverages, and combines several lines in the literature:

• Spectral and operator perspectives on self-supervised feature learning [haochen2021spectral, balestriero2022recover, zhai2025contextures].
• Information-theoretic and algorithmic stability-based generalization/label efficiency bounds.
• Classical graph-Laplacian SSL results, especially the fast-rate mechanism from [ZhangLaplacian].
• Prior empirical/analytic measures of augmentation quality (label boundary crossing, augmentation complexity) now placed inside a sample-efficient rate bound.

Future Directions

The theoretical framework opens multiple research axes:

• Extension to inductive settings and generalization to unseen samples.
• Characterization of realistic augmentation distributions under which strong balance/margin conditions hold.
• Algorithmic variants implementing truly label-independent regularizers, possibly with adaptive kernel estimation.
• Application to graph and manifold-constrained SSL, including node classification in real-world networks.
• Investigation of practical implications for backbone pretraining protocols and augmentation strategies beyond image domains.

Conclusion

The paper rigorously connects data augmentation-induced graph regularization to sample-efficient semi-supervised classification, analytically obtaining fast $g$8 transductive rates fundamentally governed by augmentation quality. The results bridge previously infinite-data-level spectral feature analyses and downstream finite-data performance, providing actionable guidance for design and evaluation of SSL algorithms. The theoretical machinery is directly extensible and clarifies longstanding label efficiency phenomena in representation learning (2607.07513).