Multi-Margin N-Pair Loss for Ordinal Classification

• The paper introduces a multi-margin N-pair loss that uses learnable Softplus margins for adjacent class pairs, encoding ordinal misclassification costs.
• The method employs a two-phase training protocol with joint optimization followed by margin freezing, ensuring stable convergence and improved risk calibration.
• Empirical results show 5–20% accuracy gains over standard losses, highlighting its effectiveness in high-stakes applications such as medical diagnosis.

The multi-margin N-pair (MMNP) loss is a supervised contrastive learning objective designed specifically for ordinal classification tasks, where class labels are ordered and the semantic distance between classes is not uniform. In contrast to conventional N-pair or triplet losses—which enforce a uniform separation between classes—the MMNP loss introduces a set of learnable, pair-specific margins between each pair of adjacent class ranks, allowing the embedding space to respect ordinal structure and domain-specific misclassification costs. This construction enables more interpretable, ordered representations and improved risk calibration, particularly in domains such as medical diagnosis, where certain boundaries are of critical importance (Pitawela et al., 22 Apr 2025).

1. Mathematical Formulation

Let $C$ denote the number of ordered classes $\{1,2,\dots,C\}$, $(x_i, y_i)$ a data-label pair with $y_i \in \{1,\dots,C\}$, and $z_i = e_\phi(x_i) \in \mathbb{R}^d$ the embedding of $x_i$ by encoder $e_\phi$. The cosine similarity between two normalized embeddings $z, u$ is $s(z,u)=\frac{z^\top u}{\|z\|\|u\|}$. Define a set of $C-1$ learnable margins $\{1,2,\dots,C\}$0 for each adjacent class pair.

For any classes $\{1,2,\dots,C\}$1, the cumulative margin is

$\{1,2,\dots,C\}$2

so misclassifications farther from the true class incur larger penalties.

Given a mini-batch, for each anchor $\{1,2,\dots,C\}$3:

• $\{1,2,\dots,C\}$4 (positives),
• $\{1,2,\dots,C\}$5 (negatives),

the MMNP loss for the anchor is

$\{1,2,\dots,C\}$6

This generalizes N-pair or triplet loss by replacing a global margin with a class-pair–specific cumulative margin $\{1,2,\dots,C\}$7, encoding the ordinal misclassification structure (Pitawela et al., 22 Apr 2025).

2. Learning and Parametrization of Margins

Each margin $\{1,2,\dots,C\}$8 is a learnable non-negative scalar. To ensure positivity and facilitate optimization, they are parametrized using the Softplus function:

$\{1,2,\dots,C\}$9

This guarantees $(x_i, y_i)$0 and smooth gradients. Margins may optionally be constrained to be above a lower bound $(x_i, y_i)$1 via penalty terms, but in practice $(x_i, y_i)$2 is sufficient.

Initialization is performed by sampling $(x_i, y_i)$3, resulting in initial margins in $(x_i, y_i)$4. A larger $(x_i, y_i)$5 signifies a more “costly” error across the $(x_i, y_i)$6 class boundary, enforcing a larger separation in embedding space at that transition. Learned margins thus directly encode the domain-informed difficulty or consequence of adjacent class confusion, enabling interpretable and controlled separation (Pitawela et al., 22 Apr 2025).

3. Training Protocol

Training employs a two-phase schedule to ensure effective optimization and stable convergence:

Phase 1: Joint Learning

1. Mini-batches are constructed by sampling at least two distinct classes with $(x_i, y_i)$7 samples per class.
2. For each anchor $(x_i, y_i)$8 in the batch:
   • Compute embedding $(x_i, y_i)$9,
   • Form $y_i \in \{1,\dots,C\}$0 and $y_i \in \{1,\dots,C\}$1,
   • Compute $y_i \in \{1,\dots,C\}$2 using the double sum and Softplus margins,
   • Compute standard cross-entropy loss $y_i \in \{1,\dots,C\}$3.
3. Accumulate total loss as

$y_i \in \{1,\dots,C\}$4

(with $y_i \in \{1,\dots,C\}$5).

1. Update encoder, classifier, and margins $y_i \in \{1,\dots,C\}$6 via Adam optimizer.

Phase 2: Margin Freezing

1. Freeze all $y_i \in \{1,\dots,C\}$7 and margins $y_i \in \{1,\dots,C\}$8.
2. Continue updating encoder and classifier using only $y_i \in \{1,\dots,C\}$9.
3. Early stopping is triggered after 10 epochs without validation improvement.

Cosine similarity enforces $z_i = e_\phi(x_i) \in \mathbb{R}^d$0 normalization of embeddings inherently. No additional weight decay is needed beyond the Adam defaults. The Softplus parametrization prevents margin collapse to zero (Pitawela et al., 22 Apr 2025).

4. Theoretical and Empirical Properties

Gradient magnitudes in MMNP scale with ordinal distance: negatives from more distant classes produce larger gradients (due to larger $z_i = e_\phi(x_i) \in \mathbb{R}^d$1), while nearby negatives are weakly repelled. This aligns embedding geometry with the cost structure of ordinal mistakes.

The two-phase schedule and Softplus margins mitigate trivial solutions (e.g., all margins vanishing). Empirically, training converges stably within a few hundred epochs. Margins $z_i = e_\phi(x_i) \in \mathbb{R}^d$2 are interpretable as quantitative measures of boundary importance; embedding visualizations such as UMAP show ordered class layouts with proportional gaps corresponding to learned margins.

Ablation studies indicate:

• Fixing all margins to 1 degrades accuracy by 1–3%.
• Using a single shared learnable margin underperforms full MMNP by 2–4%.
• Employing the two-phase schedule with frozen margins (Phase 2) consistently yields an additional +1–2% accuracy.
• Controlling critical margins (e.g., enforcing a larger gap at a medical decision boundary) can substantially reduce clinically harmful errors, e.g., a 15–20% reduction in misclassifications at the pre-cancerous vs. cancerous threshold, with only modest overall accuracy costs.

On five diverse ordinal classification benchmarks, MMNP consistently outperforms:

• Standard cross-entropy (by 5–20% accuracy gain),
• Supervised contrastive loss (by 3–10%),
• Single-margin N-pair loss (by 2–5%), and
• Other state-of-the-art ordinal methods (by 2–8%).

Robustness to label-bias noise is also demonstrated; for instance, on simulated bias between grades 1 and 2, CLOC with a fixed critical margin recover +7.9% accuracy over GOL, the next-best method (Pitawela et al., 22 Apr 2025).

5. Hyperparameter and Implementation Considerations

Key recommendations include:

• Batch size: 32–64, ensuring at least two classes per batch and $z_i = e_\phi(x_i) \in \mathbb{R}^d$3 samples per class.
• Embedding dimension: use encoder default (e.g., $z_i = e_\phi(x_i) \in \mathbb{R}^d$4 for ResNet50).
• Optimizer: Adam, learning rate $z_i = e_\phi(x_i) \in \mathbb{R}^d$5, no warmup, zero weight decay.
• Margin parametrization: $z_i = e_\phi(x_i) \in \mathbb{R}^d$6 (Softplus),
• Lower margin bound $z_i = e_\phi(x_i) \in \mathbb{R}^d$7: typically 0 or a small positive value (e.g., 0.1),
• Epochs: Phase 1 for up to 300–500 epochs (early stop on accuracy or collapse), Phase 2 for 100–200 epochs,
• Loss weight $z_i = e_\phi(x_i) \in \mathbb{R}^d$8: 1.0 (tune in $z_i = e_\phi(x_i) \in \mathbb{R}^d$9 as needed).

This structured protocol provides reliable convergence, scalable to varied datasets and rank structures (Pitawela et al., 22 Apr 2025).

6. Applications and Broader Context

The MMNP loss is directly beneficial in ordinal settings where domain-specific class boundaries warrant differentiated treatment. In clinical decision-making—such as tumor staging and disease grading—errors crossing certain boundaries can have disproportionate consequences. MMNP enables these considerations to be encoded directly into the embedding space, facilitating both improved classification and actionable interpretability.

On image datasets such as Adience (age), Historical Colour Image Dating, Knee Osteoarthritis, Indian Diabetic Retinopathy (IDRID), and Breast Carcinoma Subtyping (BRACS), MMNP as implemented in CLOC achieves state-of-the-art performance, scalable to both real-world and synthetic medical bias scenarios. The flexibility to fix or learn margins at critical boundaries permits integration with domain expertise, reducing sensitivity to dataset-specific label bias and improving reliability for high-stakes applications (Pitawela et al., 22 Apr 2025).

7. Summary Table: Core Loss Structure

Term	Description	Role in MMNP Loss
$x_i$0	Learnable Softplus-parametrized margin for class pair	Sets penalty for adjacent-class confusions
$x_i$1	Cumulative margin between ranks $x_i$2 and $x_i$3	Margin increases with ordinal distance
$x_i$4	Double-sum hinge loss with cumulative margin	Drives ordered, cost-sensitive embeddings

These components collectively enforce ordered representations and cost-aligned class separation, establishing MMNP as a principled extension of the N-pair loss for ordinal prediction settings (Pitawela et al., 22 Apr 2025).