---
title: Homomorphic Self-Supervised Learning
url: https://www.emergentmind.com/topics/homomorphic-self-supervised-learning-h-ssl
type: topic
---

# Homomorphic Self-Supervised Learning

Homomorphic Self-Supervised Learning (H-SSL) is a general framework for self-supervised representation learning that unifies augmentation-based and augmentation-free paradigms through the lens of equivariant (homomorphic) encoders. By design, H-SSL subsumes popular contrastive objectives and many traditional self-supervised losses when the feature extractor is augmentation-homomorphic with respect to a group of data transformations [2211.08282].

## 1. Formal Definition and Theoretical Foundation

Let $X$ denote the input space (e.g., images), and let $G$ be a discrete or continuous group of "augmentations" (e.g., translations, rotations, scalings). Consider a feature extractor $f: X \rightarrow \mathcal{H}$ and a group representation $\rho: G \rightarrow GL(\mathcal{H})$. $f$ is augmentation-homomorphic (equivariant) if, for all $g \in G$ and $x \in X$,
\[
f(T_g[x]) = \rho(g) f(x)
\]
where $T_g[x]$ denotes the action of $g$ on $x$ in input space, and $\rho(g)$ is its corresponding "lifted" action in feature space. Since $\rho$ preserves the group structure, $f$ is a homomorphism from $(G,T)$ to $(\mathrm{Aut}(\mathcal{H}), \circ)$.

This formulation enables H-SSL to operate directly in representation space using group-structure-respecting operations, rather than requiring explicit data augmentations.

## 2. Derivation of Self-Supervised Losses within H-SSL

Standard augmentation-based self-supervised learning (A-SSL)—such as SimCLR—uses contrastive objectives by sampling paired augmented versions of inputs. For a batch $\{x_i\}_{i=1}^N$, with two independent augmentations $g_1, g_2 \sim G$ and projection head $h$, the SimCLR loss is
\[
\mathcal{L}_{\text{A-SSL}}
 = -\frac{1}{N}\sum_{i=1}^N \mathbb{E}_{g_1,g_2\sim G} 
 \log\frac{ 
 \exp(\mathrm{sim}(h(f(T_{g_1}[x_i])), h(f(T_{g_2}[x_i])) )/\tau) 
 }{ 
 \sum_{k\neq i}\sum_{(j,\ell)\in\{1,2\}^2} 
 \exp(\mathrm{sim}(h(f(T_{g_j}[x_i])), h(f(T_{g_\ell}[x_k])) )/\tau) 
 }
\]
When $f$ is $G$-equivariant, this loss can be restated purely in feature-space by leveraging the homomorphic property:
\[
h(f(T_g[x])) = h(\rho(g)f(x))
\]
Each input thus yields a single feature map $Z_i = f(x_i)$, with "fibers" $Z_i(g) = \rho(g^{-1}) Z_i(e)$. H-SSL positives are constructed by sampling fibers $g_1^{-1} \cdot \Omega_0$ and $g_2^{-1} \cdot \Omega_0$ from $Z_i$, with $\Omega_0 \subseteq G$ a base-set of prescribed size.

The H-SSL loss is then:
\[
\mathcal{L}_{\text{H-SSL}}
 = -\frac{1}{N}\sum_{i=1}^N \mathbb{E}_{g_1,g_2\sim G} 
 \log\frac{ 
 \exp(\mathrm{sim}(h(Z_i(g_1^{-1}\cdot\Omega_0)), h(Z_i(g_2^{-1}\cdot \Omega_0)) )/\tau) 
 }{ 
 \sum_{k\neq i}\sum_{(j,\ell)\in\{1,2\}^2} 
 \exp(\mathrm{sim}(h(Z_i(g_j^{-1}\cdot\Omega_0)), h(Z_k(g_\ell^{-1}\cdot\Omega_0)) )/\tau) 
 }
\]
This form is algebraically identical to the A-SSL loss under equivariance, rendering A-SSL a special case of H-SSL.

Selection of $G$ and $\Omega_0$ recovers various objectives; for instance, $G$ as the spatial-translation group with $\Omega_0$ a single location reduces to the local DIM(L) or Greedy InfoMax objectives, and CPC emerges for appropriate choices of heads.

## 3. Equivalence Conditions and Failure Modes

**Proposition 3.1** states that if $f$ is exactly $G$-equivariant, the A-SSL and H-SSL losses are *identical* under fiber sampling in feature-space. The proof consists of substituting $f(T_g[x]) = \rho(g)f(x)$ and re-indexing over the group $G$.

**Corollary 3.2** establishes a critical failure: if $f$ is not homomorphic—i.e., $T_g$ and $f$ do not commute—then H-SSL cannot simulate the effect of input augmentations. Empirically, this results in representations that collapse or perform at chance level, in contrast to A-SSL, which still benefits from explicit input transformations.

## 4. Empirical Evaluation

Experimental validation employed three groups: rotation (four $90^\circ$ steps), translation (±20% shifts), and scale (six downscaling factors). Equivariant backbones were used for each group (e.g., rotation-equivariant CNN, standard CNN, SESN).

- **Datasets:** MNIST, CIFAR-10, Tiny ImageNet.
- **Analysis:** Linear-probe accuracy after SSL pretraining was measured for both A-SSL (with explicit augmentation) and H-SSL (feature-space only).

Key findings are summarized in the following table:

| Augmentation Group | CIFAR-10 (A-SSL) | CIFAR-10 (H-SSL) |
|--------------------|------------------|------------------|
| Translation        | 39.2 ± 0.5%      | 36.3 ± 1.1%      |

*This close correspondence confirms the theoretical equivalence of A-SSL and H-SSL when equivariant architectures are used.*

When replacing equivariant layers with generic MLPs or non-equivariant CNNs, H-SSL performs at random or "frozen" levels, demonstrating the necessity of equivariance for feature-space contrastive pairs.

## 5. Relationship to Augmentation-Based SSL and Parameter Space

H-SSL introduces two key hyperparameters absent from vanilla A-SSL:

- The base-set size $|\Omega_0|$, determining how many fibers are grouped as a "view."
- The topographic distance between $g_1$, $g_2$ in $G$, controlling the effective "augmentation strength."

Empirical variation shows that increasing $|\Omega_0|$ interpolates smoothly from local (DIM(L)) to global (SimCLR) losses, yielding only minor changes in downstream linear probe accuracy (within ±2% on CIFAR-10). Similarly, increasing the maximum allowable topographic distance $\|g_1 - g_2\|$ improves then degrades performance, paralleling findings in A-SSL regarding the utility of strong augmentations.

Temperature $\tau$, embedding dimensionality, and projection head size remain as in SimCLR, confirming that H-SSL generalizes existing A-SSL setups.

## 6. Broader Implications and Directions

Homomorphic Self-Supervised Learning offers several conceptual and practical advantages:

- **Unified Perspective:** Provides a principled bridge between augmentation-based and augmentation-free SSL, subsuming contrastive, alignment, uniformity, and local InfoMax variants as instantiations of a single group-equivariant InfoNCE objective.
- **Generalization:** Admits any $G$ for which an equivariant feature extractor is available, potentially allowing for multi-scale, multi-orientation, or learned, data-driven symmetries.
- **Novel Design Axes:** Enables new forms of view sampling and hyperparameter tuning beyond batch size and temperature.

Current limitations include the construction of backbones that are equivariant to arbitrary data augmentations; most group convolutional networks are limited to $E(n)$ or compact Lie groups. Approaches such as learned or approximate homomorphisms (e.g., topographic VAEs, NPTNs, L-convolutions) may be needed to extend H-SSL to the diverse augmentations employed in large-scale vision tasks.

Future research directions include hybrid models combining A-SSL (where equivariance is weak) with H-SSL (where group structure is strong), learning group representations $\rho(g)$ jointly with $f$, and adapting H-SSL to Transformer architectures by constructing permutation- or patch-equivariant layers.

In summary, H-SSL reframes the diversity of contemporary self-supervised learning objectives as the outcome of constraining architectures to respect the underlying group structure of augmentations, with the InfoNCE loss serving as a universal objective across these contexts [2211.08282].

Source: https://www.emergentmind.com/topics/homomorphic-self-supervised-learning-h-ssl