Self-Supervised Label Discovery

Updated 6 July 2026

SS2LD is a framework that converts self-supervised or weak structures into label-like supervision for various downstream tasks.
It employs a canonical pipeline including candidate generation, representation learning, structure induction, and model refinement.
Empirical studies show SS2LD boosts performance in EEG, object detection, and semantic segmentation, especially with sparse labels.

Searching arXiv for the cited SS2LD and closely related papers to ground the article. Self-Supervised to Label Discovery (SS2LD) denotes a family of learning pipelines in which self-supervised, weakly supervised, or otherwise label-light structure is converted into label-like supervision and then reused by a downstream learner. The term is explicit in the refinement of pathological high-frequency oscillations (HFOs) from intracranial EEG, where a variational autoencoder (VAE) first learns morphology, latent embeddings are clustered to derive weak supervision, and a classifier then refines detection boundaries (Zhang et al., 19 Jul 2025). Closely related patterns recur across weakly supervised object detection, unsupervised semantic segmentation, object-centric self-supervision, concept discovery, and pseudo-soft-label dictionary learning, where representation learning, clustering or similarity structuring, latent label assignment, and downstream refinement form the central loop (Seo et al., 2022, Zadaianchuk et al., 2022, Hénaff et al., 2022, Shao et al., 2021, Xiang et al., 12 Jun 2026).

1. Definition, scope, and adjacent meanings

In the narrow sense, SS2LD names the EEG framework that refines legacy rule-based HFO detections into a set of pathological HFOs without manual event labels (Zhang et al., 19 Jul 2025). In the broader sense suggested by the surrounding literature, it refers to methods that start from unlabeled or coarsely labeled data, learn a structured representation, derive latent labels or label-like units from that representation, and feed those discoveries back into a predictive model. This broader reading is explicit in several papers that are described as highly relevant to the theme even when they do not use the term SS2LD themselves (Hénaff et al., 2022, Zadaianchuk et al., 2022).

The scope is wider than novel class discovery. Some systems discover proposal-level labels for multiple object instances; some discover image partitions into $K$ regions; some discover cluster IDs that become pixelwise pseudo-labels; some discover pseudo soft labels on a hypergraph; some discover natural-language concepts; and some discover new supervision rules rather than new semantic classes (Seo et al., 2022, Hénaff et al., 2022, Zadaianchuk et al., 2022, Shao et al., 2021, Xiang et al., 12 Jun 2026, Lang et al., 2020). This suggests that SS2LD is better understood as an umbrella pattern for turning structure into supervision than as a single fixed algorithm.

Several adjacent formulations clarify the boundary of the term. "Self-Supervised Self-Supervision" discovers new virtual evidences for a fixed latent label space rather than new semantic classes (Lang et al., 2020). "Self-supervised Label Augmentation via Input Transformations" expands the supervised label space to the joint space $(y,j)$ , so it is a form of automatic label refinement rather than unlabeled class discovery (Lee et al., 2019). Multi-Label Self-supervised Learning assigns multiple binary pseudo-labels by top- $k$ retrieval against a queue, but those labels are dictionary-slot activations rather than stable global semantic categories (Zhu et al., 2023). In computational drug repositioning, self-supervision addresses label sparsity by improving the latent drug representation used for downstream label prediction, which is highly relevant to SS2LD but still operates within a fixed prediction space (Yang et al., 2022).

2. Canonical pipeline

A recurring SS2LD pipeline begins with a candidate generator. In weakly supervised object detection, the candidates are region proposals, typically about 2,000 per image, produced by Selective Search or MCG (Seo et al., 2022). In unsupervised semantic segmentation, the candidates are salient object crops obtained from an unsupervised saliency model (Zadaianchuk et al., 2022). In label-free 3D object segmentation, the candidates are superpoints produced by the Felzenswalb algorithm (Zhang et al., 26 May 2026). In EEG SS2LD, the candidates are HFO events produced by legacy detectors such as STE and MNI (Zhang et al., 19 Jul 2025). In concept discovery, the candidates are natural-language hypotheses sampled from a VLLM at high temperature (Xiang et al., 12 Jun 2026). In multi-label image SSL, the candidates are localized image blocks generated by block-wise augmentation (Chen, 29 Jun 2025).

The second stage is representation learning. The representation may be proposal embeddings trained by weakly supervised contrastive loss, dense object-centric features clustered by $K$ -means, DINO object representations from cropped salient regions, VAE latents over time-frequency EEG morphology, or CLIP-grounded concept embeddings used to score textual hypotheses (Seo et al., 2022, Hénaff et al., 2022, Zadaianchuk et al., 2022, Zhang et al., 19 Jul 2025, Xiang et al., 12 Jun 2026). A closely related staged design appears in Self-Supervised Dictionary Learning, where a $p$ -Laplacian Attention Hypergraph Learning block first generates pseudo soft labels and a downstream dictionary learner then uses them as discriminative targets (Shao et al., 2021).

The third stage is structure induction. The structure may be a similarity neighborhood around a seed proposal, a within-image $K$ -means partition, a spectral clustering of object crops, a hierarchical k-means partition of VAE latents, a hypergraph-smoothed pseudo-label matrix, or a set of preference pairs built from reward differences between candidate concepts (Seo et al., 2022, Hénaff et al., 2022, Zadaianchuk et al., 2022, Zhang et al., 19 Jul 2025, Shao et al., 2021, Xiang et al., 12 Jun 2026).

The final stage is feedback into a task model. In object detection, discovered proposals are relabeled as positive pseudo groundtruths and re-enter the refinement heads and regression branch (Seo et al., 2022). In COMUS, cluster IDs are converted into semantic pseudo-masks and a DeepLabv3 model is trained on them, then self-trained again on its own predictions (Zadaianchuk et al., 2022). In EEG SS2LD, weak pathological labels are distilled into a classifier on top of the frozen encoder, and the resulting predictions are aggregated into a Resection Ratio used for surgical outcome prediction (Zhang et al., 19 Jul 2025). In S $^2$ COPE, discovered concepts are deduplicated into a concept bank and used as the basis of a logistic-regression classifier on concept activations (Xiang et al., 12 Jun 2026).

3. Forms of discovered labels

SS2LD does not discover a single universal kind of label. The discovered object can be an instance ID, a region mask, a pseudo-class, a soft label vector, a supervision rule, or a textual concept.

Discovered structure	Representative papers	Role in the pipeline
Proposal-level instance labels	(Seo et al., 2022, Pot et al., 2018)	Expand positives beyond argmax or cluster repeated instances
Region or mask labels	(Hénaff et al., 2022, Zadaianchuk et al., 2022, Zhang et al., 26 May 2026)	Partition images or scenes into object-like regions
Image-level multi-hot pseudo-labels	(Zhu et al., 2023)	Predict top- $k$ dictionary-slot activations
Pseudo soft labels	(Shao et al., 2021)	Drive label-embedded dictionary learning
Supervision rules / virtual evidences	(Lang et al., 2020)	Improve inference over fixed latent labels
Joint transformed labels $(y,j)$	(Lee et al., 2019)	Refine an existing label space via self-supervision
Natural-language concepts	(Xiang et al., 12 Jun 2026)	Build an explicit semantic vocabulary
Pathological-event weak labels	(Zhang et al., 19 Jul 2025)	Refine candidate HFO events into pathological HFOs

In computer vision, one major branch discovers object-like structure rather than stable category names. Odin discovers image partitions into $K$ regions by $(y,j)$ 0-means clustering of dense features and treats the resulting masks as latent region labels (Hénaff et al., 2022). COMUS discovers cluster IDs for salient object crops and converts those IDs into pixelwise pseudo-labels (Zadaianchuk et al., 2022). ObjSeek discovers class-agnostic object masks in 3D point clouds, so its discovered labels are pseudo-instance masks rather than named semantic categories (Zhang et al., 26 May 2026). The robotic RGB-D work clusters proposal embeddings so that each resulting cluster contains examples of one instance seen from various viewpoints and scales (Pot et al., 2018).

Another branch stays closer to classical label spaces while replacing missing labels with discovered structure. SSDL generates pseudo soft labels $(y,j)$ 1 from a hypergraph and injects them into a label-embedded dictionary learning objective (Shao et al., 2021). S4 keeps the label ontology fixed but discovers new unary and joint virtual evidences that improve inference over latent labels (Lang et al., 2020). SSLDR for drug repositioning does not discover new drug or disease classes, but it uses self-supervised consistency across SMILES and InChI views to improve the embedding used for sparse label prediction (Yang et al., 2022).

A more recent line discovers explicitly interpretable label-like units. MLS assigns multiple binary pseudo-labels by declaring the top- $(y,j)$ 2 nearest dictionary slots positive for an image crop (Zhu et al., 2023). S $(y,j)$ 3COPE discovers natural-language concepts from unlabeled images and uses cross-modal invariance and specificity as a self-supervised preference signal, producing an explicit semantic vocabulary (Xiang et al., 12 Jun 2026). This suggests that SS2LD can culminate either in unlabeled cluster IDs or in human-readable concept banks, depending on the discovery operator.

4. Core mechanisms and formal operators

One recurring mechanism is similarity-based expansion from coarse seeds. In weakly supervised object detection, the top-scoring proposal embedding $(y,j)$ 4 acts as a seed and an adaptive threshold is defined as the average dot-product similarity to the class-specific sampled embeddings $(y,j)$ 5: $(y,j)$ 6 Additional pseudo groundtruths are then mined by thresholding in the learned embedding space: $(y,j)$ 7 This discovery rule is the core reason the method expands beyond argmax labeling (Seo et al., 2022).

A second mechanism is clustering over object-centric representations. Odin applies $(y,j)$ 8-means within each image to dense features and uses the resulting clusters as region masks for object-level contrastive learning (Hénaff et al., 2022). COMUS obtains salient object crops, extracts DINO-ViT CLS features, and uses spectral clustering to assign a cluster ID $(y,j)$ 9 to each crop; the binary saliency mask $k$ 0 then becomes a semantic pseudo-mask by assigning $k$ 1 to its foreground pixels (Zadaianchuk et al., 2022). EEG SS2LD performs a hierarchical two-stage k-means procedure over VAE latent means $k$ 2: first background versus HFO-like events, then pathological versus physiological among the HFO-like events, with reconstruction loss and resection anatomy used to interpret the clusters (Zhang et al., 19 Jul 2025).

A third mechanism is graph- or factor-based latent-label inference. In SSDL, pseudo soft labels are obtained by minimizing a hypergraph smoothness objective and have the closed-form solution

$k$ 3

where $k$ 4 contains known labels for labeled samples and $k$ 5 entries for unlabeled ones (Shao et al., 2021). In S4, latent labels $k$ 6 are inferred by combining neural predictions and virtual evidences in the factorization

$k$ 7

so the discovery target is better supervision over a fixed label space rather than a new ontology (Lang et al., 2020).

A fourth mechanism uses preference optimization over concept hypotheses. In S $k$ 8COPE, candidate textual concepts $k$ 9 are scored by a cross-modal reward

$K$ 0

which rewards invariance across augmentations of the same image while penalizing generic concepts that match many unrelated images (Xiang et al., 12 Jun 2026). These scalar rewards are converted into preference pairs and optimized by DPO, so the concept proposer itself is updated by the discovery loop.

An adjacent but important mechanism is explicit label-space expansion. In self-supervised label augmentation, transformation labels are fused with supervised labels to form a joint label space $K$ 1, and the training target becomes a classifier over $K$ 2 rather than separate supervised and self-supervised heads (Lee et al., 2019). This is not unlabeled label discovery, but it shows that self-supervised variables can be treated as label refiners rather than merely auxiliary losses.

5. Empirical record across domains

The empirical literature shows that converting learned structure into labels is often more effective than relying on raw scores or direct supervision transfer alone. In weakly supervised object detection, argmax-only labeling is reported to miss roughly $K$ 3 of target objects on VOC07 and $K$ 4 on COCO14, whereas object discovery plus weakly supervised contrastive loss raises VOC07 performance from $K$ 5 mAP for Baseline OICR $K$ 6 to $K$ 7 mAP, and adaptive similarity thresholding reaches $K$ 8 versus $K$ 9 for an adaptive classification threshold (Seo et al., 2022). The same paper reports $p$ 0 mAP on VOC07 with MCG proposals, $p$ 1 on VOC12 with MCG proposals, and up to $p$ 2 AP on COCO14 with ResNet101 (Seo et al., 2022).

In unsupervised semantic segmentation, COMUS reports $p$ 3 mIoU on PASCAL VOC after two self-training iterations, far beyond previous unsupervised segmentation baselines reported there, and on COCO it reports $p$ 4 IoU over all 81 classes while discovering 34 categories with more than $p$ 5 IoU (Zadaianchuk et al., 2022). Odin reports object discovery metrics on COCO of $p$ 6, $p$ 7, and $p$ 8 when FPN is retained, and transfer results of $p$ 9 box AP and $K$ 0 mask AP with Mask R-CNN after 24-epoch fine-tuning (Hénaff et al., 2022). In label-free 3D object segmentation, ObjSeek reaches $K$ 1 AP, $K$ 2 AP@50, and $K$ 3 AP@25 on ScanNet val, outperforming the unsupervised baselines listed in the paper (Zhang et al., 26 May 2026).

In concept discovery, S $K$ 4COPE reports large gains in downstream top-1 accuracy from discovered concept banks, including BloodMNIST $K$ 5, OrganCMNIST $K$ 6, Gravity Spy $K$ 7, and iNaturalist $K$ 8, all without class labels during concept discovery or purification (Xiang et al., 12 Jun 2026). Human raters preferred S $K$ 9COPE-generated concept lists $^2$ 0 of the time with standard deviation $^2$ 1 (Xiang et al., 12 Jun 2026).

In EEG, the explicit SS2LD framework improves clinically oriented metrics on both internal and external datasets. On the Open iEEG dataset, SS2LD reports ACC $^2$ 2, F1 $^2$ 3, and SPEC $^2$ 4, outperforming eHFO and spkHFO as listed in the paper; on Zurich iEEG it reports ACC $^2$ 5, F1 $^2$ 6, and SPEC $^2$ 7 (Zhang et al., 19 Jul 2025). The ablation from weak supervision only to full SS2LD further indicates that classifier distillation and VAE-based augmentation improve the clinical proxy metrics (Zhang et al., 19 Jul 2025).

Outside vision and biosignals, SSLDR reports that the gain from self-supervision grows as labels become sparser, with average improvement over its ablation rising from $^2$ 8 on Gottlieb to $^2$ 9 on Cdataset and $k$ 0 on DNdataset, the most sparse dataset (Yang et al., 2022). This supports a broader SS2LD reading: structure-derived supervision becomes more valuable as conventional labels become less informative.

6. Distinctions, assumptions, and limitations

A central distinction is that SS2LD is not synonymous with new semantic class discovery. Odin discovers per-image region partitions, not stable semantic categories across images (Hénaff et al., 2022). S4 discovers new supervision rules for a fixed output space, not new classes (Lang et al., 2020). MLS produces dictionary-slot-level latent labels that are transient and not semantically named (Zhu et al., 2023). Self-supervised label augmentation creates transformation-indexed sublabels attached to known parent classes (Lee et al., 2019). A plausible implication is that SS2LD should be treated as a continuum ranging from label refinement to latent category induction, rather than a binary property.

Most SS2LD systems also rely on strong priors. Weakly supervised object discovery assumes good proposal recall and a reasonable top-scoring seed (Seo et al., 2022). COMUS depends heavily on the quality of unsupervised saliency and on a fixed number of clusters equal to the number of foreground classes during standard evaluation (Zadaianchuk et al., 2022). ObjSeek depends on superpoint quality, DINOv2 and TRELLIS priors, PPO, and thresholded reward modules (Zhang et al., 26 May 2026). EEG SS2LD assumes that legacy detectors operate at high recall and that resected-region concentration is a useful prior for interpreting one cluster as pathological (Zhang et al., 19 Jul 2025). SSDL depends on partial labels in $k$ 1, so its main experimental setting is semi-supervised rather than label-free in the strict sense (Shao et al., 2021).

Failure modes are correspondingly varied. Similarity-based expansion can increase recall while slightly hurting conservative localization metrics; the WSOD paper explicitly notes a recall/precision tradeoff and reports failures on statues, dolls, and fragmented boxes (Seo et al., 2022). Object-centric clustering can miss non-salient classes or collapse complex scenes into too few semantic regions (Zadaianchuk et al., 2022). Per-image $k$ 2-means partitions can merge multiple same-class instances or fail to infer the number of objects present (Hénaff et al., 2022). Queue-based pseudo-labeling can suffer from stale embeddings, false positives, and non-stationary labels (Zhu et al., 2023). Clinically guided clustering remains operational rather than ontological: EEG SS2LD avoids the need for universal pathological-HFO annotations, but it does not resolve the biological ambiguity of pathological versus physiological HFOs (Zhang et al., 19 Jul 2025).

Despite these limitations, the literature converges on a stable methodological insight. Strong representation learning alone is rarely sufficient; the decisive step is the conversion of representation geometry into a supervisory object that can be reused by the task model. That object may be a discovered proposal, a spectral cluster ID, a pseudo soft label matrix, a virtual evidence, a concept preference, or a transformation-indexed sublabel. Across domains, SS2LD denotes precisely this conversion from learned structure to usable labels.