GCA-INCE: Sinkhorn Contrastive Alignment

Updated 12 December 2025

The paper introduces a novel framework that interprets contrastive learning as an entropic optimal transport problem using multistep Sinkhorn iterations.
It refines standard InfoNCE by replacing sampled negatives with a distribution-aware optimal transport plan, leading to improved alignment and uniformity.
Multimodal extensions and cross-attention strategies demonstrate significant gains in tasks like low-resource captioning and semantic grounding.

Multistep Sinkhorn-based Contrastive Alignment (GCA-INCE) is a general framework that connects contrastive learning (CL) with distribution alignment via entropic optimal transport. By interpreting the objective of contrastive learning as an optimal transport problem over learned representations, GCA-INCE enables more distribution-aware, theoretically principled, and empirically effective self-supervised learning methods. The technique generalizes standard noise-contrastive estimation (NCE) losses by replacing sampled negative pairs with an optimal transport plan obtained through multistep Sinkhorn iterations. It has been adapted for visual, language, and multimodal domains, including applications in low-resource cross-lingual captioning, where it underpins advances in semantic alignment and grounding.

1. Mathematical Foundations: Contrastive Alignment as Entropic Optimal Transport

At the core of GCA-INCE is the formulation of contrastive alignment as an entropic optimal transport (OT) problem. Given a minibatch of $B$ samples $\{x_i\}_{i=1}^B$ , each sample is subjected to augmentations and embedded using a normalized encoder $\widetilde f_\theta(x) = f_\theta(x)/\|f_\theta(x)\|$ . The pairwise cost matrix is defined via cosine dissimilarity:

$C_{ij} = 1 - \langle \widetilde f_\theta(x_i'), \widetilde f_\theta(x_j'') \rangle.$

The entropic OT formulation seeks a transport plan $\Pi \in \mathbb{R}^{B \times B}_+$ with prescribed marginals $\mu, \nu \in \Delta_B$ (typically uniform), by solving:

$\min_{\Pi \in U(\mu, \nu)} \langle \Pi, C \rangle - \varepsilon H(\Pi), \quad H(\Pi) = -\sum_{i,j} \Pi_{ij} \log \Pi_{ij},$

where $U(\mu,\nu) = \{\Pi\,:\,\Pi 1 = \mu,\, \Pi^T1=\nu\}$ . The entropic term $\varepsilon H(\Pi)$ regularizes the transport plan, controlling the trade-off between sparsity and smoothness.

2. Loss Derivation, Sinkhorn Algorithm, and Multistep Normalization

The link between contrastive learning and OT is made explicit through the use of Sinkhorn’s algorithm. Defining $K_{ij} = \exp(-C_{ij}/\varepsilon)$ , Sinkhorn iterations alternately scale rows and columns to match the prescribed marginals:

Input: C ∈ ℝ^{B×B}, μ, ν ∈ Δ_B, ε>0, T
K = exp(−C/ε)
u = ones(B)/B; v = ones(B)/B
for t in 1…T:
   u ← μ./(K v)
   v ← ν./(K^T u)
Π^* = diag(u) K diag(v)

For

T=1

, row-normalization yields the InfoNCE scheme. As

T

increases, the transport plan

\pi^* = u K v

converges to a doubly-stochastic alignment, yielding the GCA-INCE loss:

$\mathcal{L}_{\mathrm{GCA\text{-}INCE}} = - \sum_{i,j} \pi^*_{ij} \log \frac{\exp(f_\theta(x_i') \cdot f_\theta(x_j'') / \tau)}{\sum_k \exp(f_\theta(x_i') \cdot f_\theta(x_k'') / \tau)}.$

Each additional Sinkhorn iteration tightens the plan, lowering $\mathrm{KL}(\mathcal{I} \| \Pi)$ and improving alignment and uniformity properties, with the process terminating under a convergence criterion (e.g., $\ell_\infty$ norm threshold).

3. Theoretical Properties and Computational Complexity

Sinkhorn’s algorithm exhibits linear convergence (Hilbert metric contraction), producing a unique entropy-regularized coupling $\pi^*$ . Each Sinkhorn step reduces $\mathrm{KL}(\mathcal{I}\|\Pi^{(t)})$ (where $\mathcal{I}$ is the uniform plan) and raises a uniformity metric. For batch size $B$ , each iteration costs $O(B^2)$ flops, and practical convergence is observed for $T \approx 5$ –$20$. For CIFAR-10-sized batches ( $B=1024$ ), 5 steps result in only a $\approx5\%$ increase in floating point operations over baseline InfoNCE. Gradients need only be computed through the final $\Pi^*$ , not through intermediate iterations, resulting in moderate overhead.

4. Comparison to Standard InfoNCE and Negative Sampling

In the limit of a single normalization ( $T=1$ ) and uniform marginals, GCA-INCE reduces to InfoNCE:

$\pi_{ij} = K_{ij}/\sum_k K_{ik}$

recovering the exponentiated similarity form. If the plan collapses to a diagonal (one-hot case), the loss is equivalent to fully supervised alignment. The multistep Sinkhorn plan soft-reweights all $B$ negatives rather than sampling a small number of negatives, adapting to global sample structure and yielding finer control over the penalization of near-positive negatives. This distribution-aware weighting improves both the "alignment" and "uniformity" of representations.

5. Multimodal Extensions: Fine-Grained Patch Alignment and Cross-Attention

In cross-modal applications such as Bengali image captioning, GCA-INCE is extended to combine several synergistic losses:

Patch Alignment Loss (PAL): Pools evidence only from regions attended by a language decoder, using cross-attention weights over vision model patches.
InfoNCE: Prevents trivial representational collapse, enforcing global real–synthetic separation.
Sinkhorn-based OT Regularizer: Operates at the fine-grained patch level, using attention-filtered similarity matrices and solving an OT problem with $T$ Sinkhorn steps to match grammatical and visually salient regions.
Joint objective: Combines PAL, InfoNCE, OT, and language-modeling cross-entropy in a weighted sum:

$\mathcal{L} = \mathcal{L}_{\mathrm{CE}} + \lambda \mathcal{L}_{\mathrm{PAL}} + \alpha \mathcal{L}_{\mathrm{InfoNCE}} + \beta \mathcal{L}_{\mathrm{OT}}$

with hyperparameter weighting.

This multimodal approach yields strong improvements in patch-level grounding and centroid alignment, reducing embedding gaps (e.g., a 41% reduction in real-synthetic centroid gap) and boosting metrics such as BLEU-4 and BERTScore-F1 on challenging low-resource datasets (Anonto et al., 22 Sep 2025).

6. Practical Implementation and Hyperparameters

Recommended hyperparameters, empirically validated, include:

Regularization: $\varepsilon \in [0.1, 1.0]$ , optimal near 0.2.
Sinkhorn steps: $T=5$ (standard vision tasks) up to $T=20$ ; diminishing returns beyond $T=10$ .
Batch size: $B=512$ or $1024$.
Temperature: $\tau=0.5$ (GCA-INCE); InfoNCE temperature $t=0.07$ (PAL+InfoNCE).
Models: Vision encoder (ResNet-18/50 or MaxViT), projector MLP (2048→128), mBART-50 decoder (for language).
Optimizers: SGD or Adam/AdamW with typical learning rate and momentum values.
Training tricks: Use log-domain updates to avoid numerical underflow; gradient backpropagation only through final $\Pi^*$ ; early stopping on transport plan convergence ( $\ell_\infty$ norm); batch-level mixed-precision and learning rate scheduling.

Ablation studies reveal that accuracy rises steeply from $T=1$ to $T=5$ and plateaus after $T=10$ . Uniformity and alignment measures improve monotonically with $T$ , but computational cost increases linearly in $T$ ; excessive iterations yield minimal further benefit (Chen et al., 27 Feb 2025).

7. Empirical Results and Impact

On vision benchmarks, GCA-INCE achieves consistent improvements over the InfoNCE baseline:

CIFAR-10 (ResNet-18, linear eval): InfoNCE 92.0%, GCA-INCE 93.0%, GCA-UOT 93.5%
CIFAR-100: 71.1% → 71.6% → 72.2%
SVHN: 92.4% → 92.6% → 93.8%
Under strong augmentation: GCA-UOT outperforms baselines by ≈1–2% absolute (Chen et al., 27 Feb 2025).

In multimodal, low-resource captioning, a combination of PAL, InfoNCE, and Sinkhorn OT achieves BLEU-4 of 12.29 and BERTScore-F1 of 71.20 on Flickr30k-1k, substantially outperforming both cross-entropy and InfoNCE+OT without PAL, and closing the real-synthetic embedding gap (Anonto et al., 22 Sep 2025):

Method	BLEU-4	METEOR	BERT-F1
CE (real only)	5.80	24.96	68.38
CE + synthetic images	5.91	24.51	68.69
CE + InfoNCE	7.50	24.99	69.70
CE + InfoNCE + OT	7.52	26.22	70.21
PAL	10.19	27.29	70.97
PAL + InfoNCE	10.34	27.49	70.69
PAL + InfoNCE + OT (full GCA-INCE)	12.29	27.98	71.20

Key findings include the importance of PAL for correspondence, the synergies of InfoNCE and OT for semantic fidelity, and the superior grounding and separation in embedding space provided by multistep Sinkhorn-based GCA-INCE.

GCA-INCE reframes contrastive learning as generalized alignment via entropic optimal transport, enabling tighter theoretical guarantees, structured matching, and broad extensibility to both unimodal and cross-modal domains. Its multistep Sinkhorn refinement yields provable and empirically validated gains, at modest computational cost, and provides a principled toolkit for leveraging domain-specific knowledge in representation learning (Chen et al., 27 Feb 2025, Anonto et al., 22 Sep 2025).