Soft Fusion Contrastive Learning

• The paper introduces a framework where traditional one-hot contrastive targets are replaced by adaptive soft labels to capture nuanced relational structures.
• It leverages techniques such as adaptive soft relabeling, confidence weighting, and cross-modal fusion to enhance feature learning across varied domains.
• Empirical results in vision, time series, and multimodal applications show that soft contrastive objectives improve representation quality and training robustness.

Searching arXiv for recent and foundational papers on soft/soft-fusion contrastive learning and closely related formulations. Soft Fusion Contrastive Learning denotes a family of contrastive-learning formulations in which rigid binary assignments—one positive, all others negative, or fixed fusion rules across signals—are replaced by soft similarity structure, adaptive weights, neighborhood-based positives, or differentiable mixtures of complementary objectives. In the cited literature, this idea appears both as a specific method name, as in visuomotor visuo-tactile learning, and as a broader design pattern spanning self-supervised vision, time series, multilingual alignment, graph representation learning, recommendation, Earth observation, finance, and multimodal perception (Tian et al., 12 Feb 2026, Feng et al., 2022, Denize et al., 2022). The common goal is to make the learned geometry more faithful to latent semantic, structural, temporal, or cross-modal relations than standard hard instance discrimination.

1. From hard instance discrimination to soft relational objectives

The immediate historical background is standard NCE- or InfoNCE-style contrastive learning, where two augmented views of the same instance are treated as the sole positive pair and every other sample is treated as noise. Several papers argue that this assumption is systematically too strong. In vision, semantically related images can become false negatives through class collision; in time series, nearby timestamps and shape-similar series are not equally unrelated; in multilingual alignment, non-parallel sentence pairs may still be semantically close; and in cross-modal robotics, nearby manipulation states may be semantically compatible even when they are not exact synchronized pairs (Feng et al., 2022, Denize et al., 2021, Lee et al., 2023, Park et al., 2024).

A first line of work therefore softens the target distribution itself. "Similarity Contrastive Estimation" formulates contrastive learning as cross-entropy against a continuous target similarity distribution rather than a one-hot identity target, explicitly aiming to preserve semantic relations while retaining discriminative power (Denize et al., 2021). "Adaptive Soft Contrastive Learning" makes the same move from a hard instance-discrimination label to an adaptive soft label over a memory bank, with extra positive mass assigned to neighbors according to relative similarity and confidence (Feng et al., 2022). "Soft Neighbors are Positive Supporters in Contrastive Visual Representation Learning" extends the same intuition by treating retrieved correlated instances as partial positives with continuous positiveness scores rather than categorical neighbors (Ge et al., 2023).

A second line of work extends the notion of softness beyond image instance discrimination. "Soft Contrastive Learning for Time Series" assigns weights in $[0,1]$ to both inter-series and temporal relations, while "Improving Multi-lingual Alignment Through Soft Contrastive Learning" distills teacher-defined semantic similarity into cross-lingual contrastive supervision (Lee et al., 2023, Park et al., 2024). This suggests that Soft Fusion Contrastive Learning is not a single architecture, but a recurrent response to the same deficiency: hard contrastive objectives flatten relation structure that the downstream task actually needs.

2. Core mathematical patterns

A central template is to keep the contrastive prediction distribution but replace the hard target. In ASCL, the standard MoCo/InfoNCE loss

$$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$

is rewritten as a cross-entropy

$$L = -\sum_{j=1}^{n+1} y_j \log p_j, \qquad p_j = \frac{\exp(z_q^T z'_j/\tau)}{\sum_{i=1}^{n+1}\exp(z_q^T z'_i/\tau)},$$

and the one-hot pseudo-label is replaced by an adaptive soft label derived from a relative similarity distribution $q_j$ and a confidence score

$c = 1 - \frac{H(\mathbf q)}{\log(n)}.$

The defining move is that the predictive softmax remains contrastive, but the supervision becomes neighborhood-aware and confidence-weighted (Feng et al., 2022).

SCE makes the same shift in a teacher-student momentum setting. Its target distribution is

$w^2_{ik} = \lambda \cdot \mathbbm{1}_{i=k} + (1-\lambda)\cdot s^2_{ik},$

where $s^2_{ik}$ is a sharpened similarity distribution from the target branch, and the loss is

$$L_{SCE} = - \frac{1}{N} \sum_{i=1}^N\sum_{k=1}^N w^2_{ik}\log(p^1_{ik}).$$

The paper’s decomposition

$$L_{SCE} = \lambda \cdot L_{InfoNCE} + \mu \cdot L_{ReSSL} + \eta \cdot L_{Ceil}, \qquad \mu=\eta=1-\lambda,$$

makes explicit that soft contrastive learning can be viewed as a fused objective combining hard positive matching, relational supervision, and contrast against non-positives in one cross-entropy (Denize et al., 2022).

Other domains instantiate the same pattern with different supervisory signals. In SoftCLT, soft assignments are functions of data-space distance and timestamp difference; in FASCL, the target distribution is a softmax over pairwise future return correlations and the loss is a KL divergence between target and predicted similarity distributions; in Contrastive FUSE, soft fusion is realized not as label smoothing but as a spectral objective

$$J(S) = \mathrm{Tr}(S^\top \widetilde B S) - \lambda\, \mathrm{Tr}(S^\top L_c S),$$

which blends community-aware structural coherence with signed pairwise supervision in a single quadratic form (Lee et al., 2023, Lee et al., 11 Feb 2026, Chakraborty et al., 19 May 2026).

These formulations indicate three recurring mathematical motifs. First, the positive set is expanded from a singleton to a distribution or neighborhood. Second, extra relations are weighted continuously rather than discretely. Third, many methods introduce a sharpening or confidence mechanism—through $$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$0, $$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$1, entropy-based confidence, or separate target temperatures—to suppress noisy relations rather than trusting every soft neighbor equally (Feng et al., 2022, Denize et al., 2022).

3. Fusion mechanisms beyond the loss

Soft Fusion Contrastive Learning is not restricted to redefining positives and negatives; in several systems it also changes how heterogeneous signals are integrated. ViTaS is the most literal use of the phrase. It treats vision and touch as partially aligned and partially complementary observations, replaces the conventional one-paired-sample positive with a top-$$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$2 cross-modal positive set, alternates the alignment direction across modalities, and augments the contrastive objective with a CVAE so that fused visuo-tactile representations preserve enough information to reconstruct the current image under occlusion (Tian et al., 12 Feb 2026).

Recommendation models use related mechanisms, but at the feature level rather than the modality-pair level. The cold-start recommender in "Contrastive Learning for Cold Start Recommendation with Adaptive Feature Fusion" encodes user, item, and contextual signals into a common space, computes adaptive fusion weights through an attention-style softmax, and optimizes recommendation loss jointly with an InfoNCE-style contrastive branch (Hu et al., 5 Feb 2025). SLSRec separates long-term and short-term interests, calibrates them with a triplet-style contrastive objective, and then softly fuses them by a target-aware gate

$$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$3

where $$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$4 depends on $$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$5, $$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$6, and the target item embedding $$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$7 (Zhou et al., 6 Apr 2026).

Earth observation and multimodal perception provide further variants. SoftCon builds a dense scene-to-scene similarity matrix from normalized multi-hot land-cover labels and combines a MoCo-style contrastive loss with a pairwise BCE soft similarity loss, thereby replacing exact positive matching with graded scene overlap (Wang et al., 2024). CLMLF performs token-level text-image fusion through multi-layer Transformer self-attention and shapes the fused representation space with label-based and data-based contrastive objectives (Li et al., 2022). CLLAP, although framed as LiDAR-augmented radar-camera pretraining, likewise operates at the feature level: local column-wise and global view/modality consistency losses are used to improve radar-camera feature alignment before downstream detection, rather than performing hard decision fusion (Liu et al., 27 Apr 2026).

A plausible implication is that “fusion” in this literature is polysemous. It can denote soft fusion of labels, soft fusion of neighborhoods, soft fusion of modalities, or soft fusion of objectives. What remains stable is the rejection of hard, static, or single-source supervision.

4. Representative domains and formulations

The literature is best read as a set of domain-specific realizations of the same design principle.

Domain	Representative method	Soft signal or fusion mechanism
Self-supervised vision	ASCL (Feng et al., 2022)	Adaptive soft relabeling over memory-bank neighbors with entropy-based confidence
Image and video SSL	SCE (Denize et al., 2022)	Mixture of one-hot positive mass and target-branch similarity distribution
Visual SSL with neighbors	SNCLR (Ge et al., 2023)	Top-$$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$8 retrieved neighbors with continuous positiveness scores
Time series	SoftCLT (Lee et al., 2023)	Soft assignments from data-space distance and timestamp difference
Multilingual sentence alignment	IMASCL (Park et al., 2024)	Teacher-defined soft similarity labels over translation-pair batches
Featureless graphs	Contrastive FUSE (Chakraborty et al., 19 May 2026)	Spectral fusion of modularity and signed pairwise supervision
Visuomotor learning	ViTaS (Tian et al., 12 Feb 2026)	Top-$$L = -\log\frac{\exp(z_q^T z_t/\tau)}{\exp(z_q^T z_t/\tau)+\sum_{i=1}^n \exp(z_q^T z_i/\tau)}$$9 cross-modal positives plus CVAE-based exploitation of complementarity
Earth observation pretraining	SoftCon (Wang et al., 2024)	Soft scene-to-scene similarity from multi-label land-cover overlap

The range of applications is wider than this table alone indicates. Finance uses future return correlation as the soft supervisory signal for retrieval embeddings (Lee et al., 11 Feb 2026). Continual learning introduces a hard-soft balance, combining sample-sample relations with fixed Neural Collapse prototypes in FNC$$L = -\sum_{j=1}^{n+1} y_j \log p_j, \qquad p_j = \frac{\exp(z_q^T z'_j/\tau)}{\sum_{i=1}^{n+1}\exp(z_q^T z'_i/\tau)},$$0 (Dang et al., 2024). Bioinformatics uses adaptive gating to fuse CNN and BiLSTM sequence features and couples that fusion with OHEM-driven contrastive learning in AVP-Fusion (Wen et al., 25 Dec 2025). Cold-start recommendation and long-/short-term session modeling bring the same pattern into recommender systems through adaptive feature weighting and contrastive calibration (Hu et al., 5 Feb 2025, Zhou et al., 6 Apr 2026).

This breadth suggests that Soft Fusion Contrastive Learning is better understood as a methodological family than as a single canonical benchmark setting.

5. Empirical behavior and characteristic ablations

Across visual self-supervision, soft formulations are repeatedly reported to improve both representation quality and training behavior. ASCL reaches 90.55 / 65.27 / 89.54 / 48.36 on CIFAR-10 / CIFAR-100 / STL-10 / Tiny ImageNet and 71.5% top-1 accuracy on ImageNet-1k with ResNet-50 after 200 epochs, while its ablations show that hard neighbor inclusion can slow learning and become brittle, whereas adaptive hard relabeling and adaptive soft relabeling retain a learning speed close to MoCo. The same study reports that weak augmentations for the momentum encoder and memory bank improve the reliability of the similarity distribution, and that $$L = -\sum_{j=1}^{n+1} y_j \log p_j, \qquad p_j = \frac{\exp(z_q^T z'_j/\tau)}{\sum_{i=1}^{n+1}\exp(z_q^T z'_i/\tau)},$$1 is best among the tested values (Feng et al., 2022).

SCE and SNCLR support a similar conclusion from different architectures. SCE reports 72.1% top-1 at 100 epochs, 72.7% at 200 epochs, 73.3% at 300 epochs, and 75.4% at 200 epochs with multi-crop on ImageNet linear evaluation, while intermediate $$L = -\sum_{j=1}^{n+1} y_j \log p_j, \qquad p_j = \frac{\exp(z_q^T z'_j/\tau)}{\sum_{i=1}^{n+1}\exp(z_q^T z'_i/\tau)},$$2 values around $$L = -\sum_{j=1}^{n+1} y_j \log p_j, \qquad p_j = \frac{\exp(z_q^T z'_j/\tau)}{\sum_{i=1}^{n+1}\exp(z_q^T z'_i/\tau)},$$3 to $$L = -\sum_{j=1}^{n+1} y_j \log p_j, \qquad p_j = \frac{\exp(z_q^T z'_j/\tau)}{\sum_{i=1}^{n+1}\exp(z_q^T z'_i/\tau)},$$4 outperform both pure contrastive and pure relational extremes (Denize et al., 2022). SNCLR reports 69.6\%, 72.4\%, 74.5\%, and 75.3\% top-1 on ImageNet linear evaluation with ResNet-50 at 100, 200, 400, and 800 epochs, and its ablation shows that removing positiveness weights degrades ViT-S top-1 from $$L = -\sum_{j=1}^{n+1} y_j \log p_j, \qquad p_j = \frac{\exp(z_q^T z'_j/\tau)}{\sum_{i=1}^{n+1}\exp(z_q^T z'_i/\tau)},$$5 to $$L = -\sum_{j=1}^{n+1} y_j \log p_j, \qquad p_j = \frac{\exp(z_q^T z'_j/\tau)}{\sum_{i=1}^{n+1}\exp(z_q^T z'_i/\tau)},$$6, indicating that binary neighbor labeling is inferior to soft weighting (Ge et al., 2023).

Time-series, robotics, recommendation, and finance show the same pattern in domain-specific metrics. SoftCLT improves TS2Vec from 83.0% to 85.0% average accuracy on UCR and from 71.2% to 75.1% on UEA, with additional gains in anomaly detection and transfer settings (Lee et al., 2023). ViTaS reports 91.4 average success on its 12-task simulated benchmark, versus 70.3 for MVT and 71.5 for CVT, and its ablation reports that removing Soft Fusion Contrastive Learning drops average performance from 92.5 to 54.7 (Tian et al., 12 Feb 2026). SLSRec achieves 0.9076 AUC on Taobao, 0.8599 on Tmall, and 0.8734 on Cosmetics, while removing contrastive learning causes smaller but consistent degradations and removing the short-term encoder is catastrophic (Zhou et al., 6 Apr 2026). FASCL attains 0.3837 FRC@1, 0.3557 FRC@5, 0.3414 FRC@10, and 0.3255 FRC@20, outperforming 13 baselines on future-behavior metrics and showing that future-aligned soft supervision matters more than softness alone (Lee et al., 11 Feb 2026).

A common empirical pattern therefore emerges: softening by itself is not sufficient; the quality of the soft relations, the sharpness of the target distribution, and the mechanism used to suppress noisy neighbors are usually decisive.

6. Conceptual boundaries, misconceptions, and limitations

Soft Fusion Contrastive Learning is often conflated with simple label smoothing, naive multi-positive learning, or modality concatenation. The literature distinguishes these cases sharply. ASCL does not merely add a few neighbors; it normalizes a confidence-weighted soft target over all candidates (Feng et al., 2022). SCE is not presented as “InfoNCE plus ReSSL,” but as a single soft cross-entropy that implicitly contains contrastive and relational terms (Denize et al., 2022). ViTaS argues explicitly that direct concatenation fails to exploit the complementary nature of vision and touch, especially under occlusion, and that separate encoders plus soft cross-modal positives are preferable (Tian et al., 12 Feb 2026).

Another misconception is that soft contrastive learning is necessarily view-based or augmentation-centric. Contrastive FUSE is explicitly not view-based; it is spectral, linear, and supervision-aware, with contrastive behavior expressed through a signed pairwise Laplacian rather than augmentation design (Chakraborty et al., 19 May 2026). Multilingual soft contrastive learning is likewise a distillation problem over sentence similarities rather than an instance-augmentation problem (Park et al., 2024). This suggests that “contrastive” in this family frequently means structured attraction and repulsion in the embedding geometry, not only positive/negative pairs generated by data augmentation.

The most persistent limitation is relation quality. ASCL’s BYOL experiment can hurt on CIFAR-100 with the default batch size, indicating that false positives can outweigh benefits when the batch is small and classes are numerous (Feng et al., 2022). ViTaS shows that too large a positive neighborhood hurts performance and that removing the CVAE degrades robustness, implying that alignment without complementarity is insufficient (Tian et al., 12 Feb 2026). FASCL notes that its target softmax emphasizes correlated pairs but does not explicitly distinguish uncorrelated from anti-correlated assets, and its evaluation is restricted to same-period retrieval (Lee et al., 11 Feb 2026). In continual learning, FNC$$L = -\sum_{j=1}^{n+1} y_j \log p_j, \qquad p_j = \frac{\exp(z_q^T z'_j/\tau)}{\sum_{i=1}^{n+1}\exp(z_q^T z'_i/\tau)},$$7 argues that soft sample relations alone can drift across tasks and must be balanced with hard prototype structure to reduce forgetting (Dang et al., 2024).

Taken together, these works imply a technical boundary condition for the field: softness is beneficial when the soft signal is informative, sharpened, and calibrated to the domain’s latent semantics. When the soft relation source is noisy, poorly sampled, or task-misaligned, the same mechanism can introduce false positives, unstable neighborhoods, or geometry drift. The central research problem is therefore not merely how to soften contrastive learning, but how to define, estimate, and regularize the right soft relations for each regime.