---
title: Adaptive Tolerance ASMS for Deep Metric Learning
url: https://www.emergentmind.com/topics/adaptive-tolerance-asms-at-asms
type: topic
---

# Adaptive Tolerance ASMS for Deep Metric Learning

Adaptive Tolerance Asymmetric Sample Mining Strategy (AT-ASMS) is a dynamic sample mining method designed for deep metric learning, specifically addressing the challenge of static, non-adaptive thresholding for positive and negative sample pair selection. AT-ASMS builds on the Asymmetric Sample Mining Strategy (ASMS), introducing adaptive mechanisms to regulate the mining of informative pairs on a per-batch basis and synergizing with a meta-learned loss margin for further performance improvements [2404.19282].

## 1. Rationale and Overview

In deep metric learning, models are trained by distinguishing between similar (positive) and dissimilar (negative) sample pairs, using a loss function applied to the mined pairs. Standard sample mining employs a symmetric threshold (tolerance) to designate which pairs are "hard" enough (i.e., most informative) for loss computation, but this approach leads to issues, such as filtering away too many informative positive pairs and admitting excessive redundant negative pairs.

ASMS addresses this by replacing the single threshold γ with two separate tolerances, γ_{pos} and γ_{neg}, allowing looser filtering for positives and stricter filtering for negatives. AT-ASMS extends this by adaptively regulating γ_{pos}, γ_{neg} according to the observed batch-wise ratio of mined negatives to positives, dynamically shifting these thresholds in response to training dynamics to improve sample mining balance and training efficiency [2404.19282].

## 2. Mathematical Formulation of AT-ASMS Mining

Let Φ(x) denote the L₂-normalized embedding of input x, and S_{pos}(x_i, x_j)=⟨Φ(x_i), Φ(x_j)⟩, S_{neg} for negative pairs.

### Static ASMS Filtering Conditions

- **Positive pair retained if:**
  $$
  S_{pos} < \max_{neg} S_{neg} + \gamma_{pos}
  $$
- **Negative pair retained if:**
  $$
  S_{neg} > \min_{pos} S_{pos} - \gamma_{neg}
  $$

Here, γ_{pos} is typically much larger than γ_{neg}, implementing asymmetry in mining selectivity.

### Batch-wise Pair Counting

For a batch of size B with N_{inst} samples per class:
- Number of possible positive pairs:
  $$
  N_{pos} = \frac{1}{2}(B\cdot N_{inst} - B)
  $$
- Number of possible negatives:
  $$
  N_{neg} = \frac{1}{2}(B^2 - B\cdot N_{inst})
  $$

Let n_{pos} and n_{neg} denote the counts of actual positives and negatives mined post-ASMS. Define the **imbalance ratio**:
$$
\xi = \frac{n_{neg}}{N_{pos}}
$$

### AT-ASMS Adaptive Update

Thresholds are adjusted if ξ > 1 (i.e., when negative pairs mined outnumber total possible positives), introducing a **zoom factor** κ and sigmoid σ(·):

- $$
  \hat{\gamma}_{pos} = \gamma_{pos} + \kappa\,\gamma_{pos}\,\sigma(\xi)
  $$
- $$
  \hat{\gamma}_{neg} = \gamma_{neg} - \kappa\,\gamma_{neg}\,\sigma(\xi)
  $$

This mechanism increases the positive tolerance (admitting more positives) while decreasing the negative tolerance (excluding more negatives) when imbalances arise.

## 3. AT-ASMS Algorithm Workflow

The following table summarizes the main steps and adaptive controls within AT-ASMS, as presented in [2404.19282].

| Step         | Operation Summary                                                                                | Condition/Update Logic                         |
|--------------|-------------------------------------------------------------------------------------------------|------------------------------------------------|
| 1. Initial ASMS | Filter positives using γ_{pos}, negatives with γ_{neg}                                        | As per static conditions                       |
| 2. Count     | n_{pos}, n_{neg} counts and compute ξ                                                           | ξ = n_{neg} / N_{pos}                          |
| 3. Adapt     | If ξ>1: update γ_{pos}, γ_{neg} via adaptive rule                                               | Else: leave γ_{pos}, γ_{neg} unchanged         |
| 4. Remine    | Re-mine pairs using updated (or original) thresholds                                            | Hand final pairs to loss function              |

*Editor's term*: "adaptive mining loop" refers to the sequence of steps 2–4.

## 4. Joint Margin Optimization via Meta-Learning

In parallel with adaptive mining, the Soft Contrastive loss's threshold/margin (λ) is meta-learned per iteration via a single-step gradient descent:

- **Soft Contrastive loss** (with parameters μ, ν, λ):
  $$
  L_{scon}(λ;P^+,P^-) = \frac{1}{\mu |P^+|} \sum_{(i,j)\in P^+} \log\left(1+e^{\mu(\lambda-S_{pos})}\right) + \frac{1}{\nu |P^-|} \sum_{(i,j)\in P^-} \log\left(1+e^{\nu(S_{neg}-\lambda)}\right)
  $$

- **Meta-update** (for λ with meta-step size φ):
  $$
  \hat\lambda_t = [\,\lambda_t - \varphi\, \frac{\partial}{\partial\lambda_t} L^m(\lambda_t;P^{mts})\,]_+
  $$
  with [·]_+ denoting non-negative clipping.

- **Parameter update** (network parameters θ, main learning rate ψ):
  $$
  \theta_{t+1} = \theta_t - \psi \frac{\partial}{\partial\theta_t} L^t(\hat\lambda_t;P_{pn})
  $$

This explicit adaptation of λ synergizes with AT-ASMS to address both mining and loss threshold selection, forming the Dual Dynamic Threshold Adjustment Strategy (DDTAS).

## 5. Hyperparameters and Implementation Guidelines

Practical tips and default hyperparameter settings are as follows [2404.19282]:

- Initial mining tolerances: γ_{pos} = 0.1, γ_{neg} = 0.01
- Zoom factor: κ = 0.5
- Loss parameters: λ_{init} = 0.7, μ = 2, ν = 40
- Meta-update step size: φ ≈ 1×10⁻³ (small for λ stability)
- Network learning rate: ψ ≈ 1×10⁻⁵ (Adam)
- Batch composition: B = 80, N_{inst} = 5 (N_{pos} = 160, N_{neg} ≈ 3000)
- λ is constrained to λ ≥ 0.
- Mining thresholds updated only when ξ > 1, reducing oscillatory behavior.
- Embeddings are L₂-normalized so that similarities lie in [–1,1].

## 6. Empirical Evaluation

With BN-Inception backbone (D=512), AT-ASMS and DDTAS yield the following recall@1 and NMI improvements:

| Dataset      | Baseline MS-Loss | Static ASMS | AT-ASMS      | DDTAS (AT-ASMS + meta λ) |
|--------------|------------------|-------------|--------------|--------------------------|
| CUB200       | R@1 ≃ 65.7%      | 68.0%       | 68.3% (NMI ≃ 71.1%) | 68.4% (NMI ≃ 71.0%)  |
| Cars196      | R@1 ≃ 84.1%      | 85.4%       | 86.4%        | 86.4% (NMI ≃ 73.3%)      |
| SOP          | R@1 ≃ 78.2%      | –           | –            | 78.0%                    |

AT-ASMS alone closes approximately 50% of the gap to state-of-the-art performance. The addition of the meta-learned loss margin (DDTAS) further improves results on standard benchmarks [2404.19282].

## 7. Context and Significance

AT-ASMS provides an adaptive, theoretically grounded alternative to grid search–based threshold selection for pair mining in deep metric learning. By integrating batch-level feedback on mined pair counts and dynamically tuning mining tolerances, it mitigates the problem of imbalance between positive and negative pairs, thereby facilitating more efficient and robust training. When combined with online meta-learning for the loss margin, as in DDTAS, the resulting system achieves highly competitive benchmark performance with reduced reliance on exhaustive hyperparameter tuning [2404.19282].

Source: https://www.emergentmind.com/topics/adaptive-tolerance-asms-at-asms