---
title: 'Clear-VAE: Weakly Supervised Disentanglement'
url: https://www.emergentmind.com/topics/clear-vae
type: topic
---

# Clear-VAE: Weakly Supervised Disentanglement

Searching arXiv for the target paper and closely related references to ground the article.
Clear-VAE denotes the instantiation of **CLEAR**—**Contrastive LEarning with Anti-contrastive Regularization**—in the latent space of a **Variational Auto-Encoder (VAE)**. It is designed for **weakly supervised disentanglement** of **task-relevant content** and **task-irrelevant style**, using only content or outcome labels and requiring neither style labels nor paired data. The framework is motivated by settings in which models learn **spurious associations** between content and superficial attributes, so that previously unseen combinations of content and style at test time degrade downstream performance; the paper presents CLEAR-VAE as a means to support both semantic manipulation of latent factors and more robust prediction under such shifts [2507.18794].

## 1. Problem setting and conceptual basis

Clear-VAE is formulated around a semantic partition of representations into **content features** and **style features**. In the terminology used by the source paper, content features are **task-relevant** and contain information needed for downstream prediction, whereas style features are **task-irrelevant** and capture superficial attributes such as race, sex, color, texture, background, or site artifacts. The central concern is that training data may contain associations between these two kinds of factors that do not generalize, even when they appear predictive in-distribution [2507.18794].

The motivating examples are especially explicit in healthcare. The paper states that one may wish to learn features that contain information about pathology yet are unaffected by race, sex, and other sources of physiologic variability, so that predictions remain equitable and generalizable across demographics. In this framing, Clear-VAE is not merely a latent-variable model for interpretability; it is a representation-learning approach targeted at **distribution shift** induced by changes in superficial characteristics.

A central clarification in the paper is that prior **unsupervised disentanglement** methods such as $\beta$-VAE and FactorVAE do not explicitly tie latent content to the supervised task. Consequently, content is not reliably identified in the representation, and disentanglement by itself does not necessarily improve downstream supervised performance if the learned factors fail to align with labels. The paper also contrasts its approach with **doubly supervised** methods that require labels for both content and style, arguing that such style labels are usually rare. Clear-VAE is therefore positioned as a practical form of **weak supervision**: labels are used only for the task-relevant factor, while style is discouraged from carrying label information.

## 2. CLEAR objective: contrastive attraction and anti-contrastive repulsion

The CLEAR framework combines two regularization objectives with opposite roles. The first is a **contrastive learning** term applied to the content latent variable $z^{(c)}$, and the second is an **anti-contrastive** term applied to the style latent variable $z^{(s)}$ [2507.18794].

For content, CLEAR seeks to maximize the mutual information between the label $y$ and the content representation $z^{(c)}$. The paper implements this with a **Supervised Contrastive Loss**, described as based on **Soft Nearest Neighbor (SNN)** or **InfoNCE**. Given a batch $\{(x_i,y_i)\}_{i=1}^N$, the model encodes each sample into $z_i^{(c)}$ and minimizes

$$
\mathcal{L}_{\mathrm{SNN}^{(c)}}=
\frac{1}{N}\sum_{i=1}^{N}
-\log
\frac{\mathrm{pos}_i^{(c)}}{\mathrm{pos}_i^{(c)}+\mathrm{neg}_i^{(c)}},
$$

where

$$
\mathrm{pos}_i^{(c)}=
\sum_{j\neq i}\mathbb{I}[y_i=y_j]
\exp\left(\frac{\mathrm{sim}(z_i^{(c)},z_j^{(c)})}{\tau}\right),
$$

and

$$
\mathrm{neg}_i^{(c)}=
\sum_{j\neq i}\mathbb{I}[y_i\neq y_j]
\exp\left(\frac{\mathrm{sim}(z_i^{(c)},z_j^{(c)})}{\tau}\right).
$$

Here, $\mathrm{sim}(\cdot,\cdot)$ is cosine similarity and $\tau$ is the temperature. Operationally, this draws samples with the same content label together and pushes samples with different labels apart.

For style, CLEAR introduces **Pair-Switching (PS)**, an anti-contrastive penalty intended to minimize the mutual information between the label $y$ and the style representation $z^{(s)}$. The key construction is the **Pair-Switching SNN Loss**:

$$
\mathcal{L}_{\mathrm{PS\text{-}SNN}^{(s)}}=
\frac{1}{N}\sum_{i=1}^{N}
-\log
\left(
\frac{\mathrm{neg}_i^{(s)}}{\mathrm{pos}_i^{(s)}+\mathrm{neg}_i^{(s)}}
\right).
$$

The paper describes this as swapping the roles of positive and negative pairs relative to the content loss. Minimizing this term encourages style representations for samples with the same label **not** to cluster together, thereby removing label information from style. This makes CLEAR distinctive among weakly supervised disentanglement methods: instead of learning style indirectly through reconstruction alone, it explicitly regularizes style to be **uninformative for prediction**.

## 3. VAE instantiation and learning objective

Clear-VAE realizes CLEAR within a VAE latent space by factorizing the encoder output into content and style components. The encoder maps $x \mapsto (z^{(c)}, z^{(s)})$ and models

$$
q_\phi(z\mid x)=q_\phi(z^{(c)}\mid x)\,q_\phi(z^{(s)}\mid x),
$$

with the two components treated as independent [2507.18794]. The decoder reconstructs $x$ from the concatenated latent vector $(z^{(c)}, z^{(s)})$.

The full training objective is

$$
L = L_{\mathrm{VAE}(\beta)}
+ \alpha_1\,\mathcal{L}_{\mathrm{SNN}^{(c)}}
+ \alpha_2\,\mathcal{L}_{\mathrm{PS\text{-}SNN}^{(s)}},
$$

where $L_{\mathrm{VAE}(\beta)}$ is the usual $\beta$-VAE loss:

$$
L_{\mathrm{VAE}(\beta)} =
-\mathbb{E}_{q_\phi(z\mid x)}[\log p_\theta(x\mid z)]
+ \beta \cdot \mathrm{KL}\big(q_\phi(z^{(c)}\mid x)\parallel p(z^{(c)})\big)
+ \beta \cdot \mathrm{KL}\big(q_\phi(z^{(s)}\mid x)\parallel p(z^{(s)})\big).
$$

Within this objective, $\mathcal{L}_{\mathrm{SNN}^{(c)}}$ encourages $z^{(c)}$ to be predictive of $y$, while $\mathcal{L}_{\mathrm{PS\text{-}SNN}^{(s)}}$ encourages $z^{(s)}$ to be independent of $y$. The coefficients $\alpha_1$ and $\alpha_2$ are weighting parameters and are described as typically set equal for simplicity.

Architecturally, the paper notes that the encoder and decoder use **standard convolutional layers**, with details deferred to supplementary material. It also emphasizes an implementation distinction: **no additional branches or paired data are required**, and **no discriminator or auxiliary network is needed** for PS-based mutual-information minimization. The stated implication is faster and simpler training relative to adversarial or variational mutual-information estimators. This suggests that Clear-VAE is intended as a comparatively direct alternative to methods that enforce invariance through extra estimation networks.

## 4. Theoretical characterization of Pair-Switching

A central theoretical claim of the paper is that the PS loss provides an **upper bound** on the mutual information between the style representation and the content label [2507.18794]. Specifically, the appendix is said to prove that

$$
\mathcal{L}_{\mathrm{PS\text{-}SNN}^{(s)}} \geq I(y, z^{(s)}) - \log N.
$$

Under this result, minimizing the PS loss minimizes an upper bound on $I(y; z^{(s)})$. In the paper’s interpretation, this makes the anti-contrastive regularizer a direct mechanism for suppressing spurious label information in style.

This theoretical perspective differentiates Clear-VAE from disentanglement approaches that rely on total-correlation penalties or from supervised invariance methods that approximate mutual information through adversarial training or auxiliary estimators. The paper further states that the anti-contrastive PS term is **as effective as more complex mutual-information estimators, but simpler and more stable**. Because no auxiliary estimator is introduced, the optimization target remains integrated into the primary latent geometry rather than delegated to a separate critic.

A related clarification concerns what the paper treats as disentanglement. The goal is not solely latent-factor independence in an unsupervised sense, but a supervised asymmetry: content should retain information about $y$, whereas style should discard it. In that sense, Clear-VAE is organized around **task-aligned disentanglement** rather than generic factor decomposition. This suggests that its notion of disentanglement is operationally tied to downstream robustness, not only to factor interpretability.

## 5. Empirical behavior: disentanglement, manipulation, and OOD prediction

The paper evaluates Clear-VAE on **Styled-MNIST, Colored-MNIST, PACS, CelebA, and Camelyon17-WILDS** [2507.18794]. The experiments are designed both to inspect the semantics of the learned latent variables and to test downstream prediction when content-style associations shift at evaluation time.

On the generative side, the paper reports that Clear-VAE allows **swapping** and **interpolation** of content and style between any pair of samples. In swapping experiments, combining the content code $z^{(c)}$ of one sample with the style code $z^{(s)}$ of another yields decoded images with content from the first and style from the second. In interpolation experiments, the model produces smooth transitions in either content or style while holding the other fixed. The qualitative outcome is described as semantically meaningful manipulation of factors such as digit identity, gender, background, color, and texture, with **no leakage across factors**.

For downstream classification, the training and evaluation protocol uses observed combinations of $(\text{content}, \text{style})$ during training and **unseen style-content combinations** at test time, i.e., an **out-of-distribution (OOD)** setting. The paper states that CNN and vanilla VAE baselines suffer large drops in **accuracy**, **AUROC**, and **Average Precision** on these OOD splits. By contrast, Clear-VAE achieves the **highest performance or close to best** across datasets and is described as particularly robust to style shift because content is not contaminated with style.

The paper also reports **state-of-the-art OOD generalization** on challenging pathology and domain generalization datasets such as Camelyon17-WILDS, where Clear-VAE is said to outperform other VAE variants and specifically tailored domain generalization methods. Since the article’s source does not reproduce the full comparison table, the exact margin is not specified here, but the stated finding situates Clear-VAE as a method for predictive robustness rather than only latent controllability.

## 6. Quantitative disentanglement, ablations, and significance

For quantitative evaluation of disentanglement, the paper uses the **Group Mutual Information Gap (gMIG)**, defined as

$$
\mathrm{gMIG}(y)=
\frac{1}{H(y)}
\left(
\frac{1}{d_c}\sum_{j=1}^{d_c} I(z_j^{(c)};y)
-
\frac{1}{d_s}\sum_{j=1}^{d_s} I(z_j^{(s)};y)
\right).
$$

This metric measures how strongly the content variables encode the label relative to the style variables. The paper states that Clear-VAE achieves **much higher gMIG** than plain $\beta$-VAE and is competitive with or better than other disentangling approaches. It further summarizes the observed values by stating that Clear-VAE achieves **gMIG $\approx 0.3$–$0.4$** on various datasets [2507.18794].

Ablation studies are used to argue that **both** regularization components are necessary. Removing either the contrastive content term or the anti-contrastive style term reduces separation and clustering structure in latent space, as illustrated by t-SNE plots and decreases in gMIG. The paper’s conclusion is explicit: both terms are essential for full semantic disentanglement and robust generalization.

The broader significance claimed by the paper centers on **fairness and robustness**. Because the method seeks to make predictions invariant to style, it is presented as supporting equitable performance across subpopulations and as being practical in real-world settings where only outcome labels are available. A plausible implication is that Clear-VAE belongs to a line of methods that treat disentanglement not as an end in itself but as a mechanism for controlling spurious correlations under domain shift. At the same time, the paper’s claims remain specific to the reported image benchmarks and to the supervised signal provided by content labels; Clear-VAE is not presented as a fully unsupervised disentanglement method, and its guarantees are tied to the assumptions encoded in the CLEAR objective.

Source: https://www.emergentmind.com/topics/clear-vae