---
title: 'SphereFace, CosFace, and ArcFace: Angular Margin Loss'
url: https://www.emergentmind.com/topics/sphereface-cosface-and-arcface
type: topic
---

# SphereFace, CosFace, and ArcFace: Angular Margin Loss

SphereFace, CosFace, and ArcFace are foundational angular-margin softmax losses designed to enhance intra-class compactness and inter-class separability for deep face recognition. Under a unified hyperspherical framework, each variant differentiates itself through the specific manipulation of angular margins in the normalized softmax cross-entropy loss, directly impacting the learning geometry and stability of face representation models. Their effect is controlled via margin functions applied in the angular space between $L_2$-normalized features and class-wise weight vectors, inducing target disambiguation in an open-set identification regime [2509.19753][2109.05565].

## 1. Unified Hyperspherical Loss Framework

These methods operate within the normalized softmax loss, where both feature vectors and class proxies are $L_2$-normalized onto a hypersphere. The loss can be written as:
\[
\mathcal{L} = -\frac{1}{N}\!\sum_{i=1}^{N} \log \frac{\exp(s\,\psi(\theta_{y_i}))}{\exp(s\,\psi(\theta_{y_i}))+\sum_{j \neq y_i}\exp(s\,\eta(\theta_j))}
\]
where:
- $\theta_j$ is the angle between feature $\mathbf{x}_i$ and weight vector $\mathbf{W}_j$, $\|\mathbf{x}_i\| = \|\mathbf{W}_j\| = 1$
- $s$ is a learnable scaling factor
- $\psi$ and $\eta$ are angular activation functions for the target and non-target classes, respectively

A margin function $\Delta(\theta) = \eta(\theta) - \psi(\theta)$ encapsulates the additional angular penalty. The sufficient margin condition $\Delta(\theta) > 0$ for $\theta \in (0, \pi]$ enforces compactness and class separability [2109.05565].

## 2. SphereFace: Multiplicative Angular Margin

**Formulation:** SphereFace imposes a multiplicative angular margin, replacing $\theta$ with $m_s \cdot \theta$ in the cosine similarity:
\[
\psi_{\mathrm{Sphere}}(\theta_{y_i}) = (-1)^k \cos(m_s \theta_{y_i}) - 2k,
\]
for $\theta_{y_i} \in [k \pi / m_s, (k+1) \pi / m_s],\ k \in \mathbb{N}$, ensuring strict monotonicity.

**Decision Boundary:** The classification rule $m_s \theta_y = \theta_j$ geometrically increases the effective separation between classes on the hypersphere.

**Characteristics:**
- The similarity curve $T_{\mathrm{Sphere}}(\theta) = \cos(m_s \theta)$ oscillates and requires piecewise correction for monotonicity.
- The gradient $\frac{dT}{d\theta} \propto -m_s \sin(m_s \theta)$ exhibits multiple zero-crossings, leading to training instability, especially for large $m_s$.
- Empirically, training with large $m_s$ can cause oscillation and collapse, requiring stabilization techniques [2509.19753][2109.05565].

## 3. CosFace: Additive Cosine Margin

**Formulation:** CosFace introduces an additive margin subtracted outside the cosine function:
\[
\psi_{\mathrm{Cos}}(\theta_{y_i}) = \cos(\theta_{y_i}) - m_c
\]

**Decision Boundary:** The rule $\cos \theta_y - m_c = \cos \theta_j$ forces a fixed angular gap between classes, with the gap magnitude set by $m_c$.

**Characteristics:**
- The similarity curve is a vertical shift, $T_{\mathrm{Cos}}(\theta) = \cos(\theta) - m_c$, which remains monotonic for $\theta \in [0,\pi]$.
- The gradient $\frac{dT}{d\theta} = -\sin \theta$, unaffected by $m_c$.
- Offers high training stability, but the penalty is spatially uniform and insensitive to intra-class sample positions [2509.19753][2109.05565].

## 4. ArcFace: Additive Angular Margin

**Formulation:** ArcFace enforces an angular margin by augmenting the argument inside the cosine:
\[
\psi_{\mathrm{Arc}}(\theta_{y_i}) = \cos(\theta_{y_i} + m_a)
\]

**Decision Boundary:** The rule $\cos(\theta_y + m_a) = \cos \theta_j$ translates to $\theta_y + m_a = \theta_j$ (principal range), reflecting an angular offset for the target class.

**Characteristics:**
- The similarity curve is a left-shift, $T_{\mathrm{Arc}}(\theta) = \cos(\theta + m_a)$.
- Monotonicity is preserved only for $m_a < \pi$; if $m_a$ is too large, non-monotonic intervals and negative gradients appear.
- The gradient $\frac{dT}{d\theta} = -\sin(\theta + m_a)$ may introduce conflicting gradient directions for certain parameterizations [2509.19753][2109.05565].

## 5. Comparative Analysis

The table below summarizes the core properties of SphereFace, CosFace, and ArcFace based on their penalty mechanisms, similarity curves, and stability:

| Method      | $T(\theta)$                             | Penalty Pattern (θ∈[0,π])                  | Stability Features                       |
|-------------|----------------------------------------|---------------------------------------------|------------------------------------------|
| SphereFace  | $\cos(m_s\theta)$ (piecewise corrected)| Small penalty near $\theta$$\approx$0, large near $\theta$$\approx$$\pi$| Oscillating similarity & gradient; unstable for large $m_s$ |
| CosFace     | $\cos\theta - m_c$                     | Uniform penalty                             | Stable, but focus fixed at $\theta$$\approx$$\pi/2$ |
| ArcFace     | $\cos(\theta + m_a)$                   | Roughly uniform; endpoint emphasis          | Monotonic only for $m_a < \pi$; negative gradients possible if $m_a$ too large  |

**Implications:** Multiplicative margins are more “geometric” and adaptive across the hypersphere, while additive margins (CosFace, ArcFace) favor operational stability but with limited flexibility in margin spatial distribution [2109.05565].

## 6. Optimization and Stability: Characteristic Gradient Detachment and Feature Normalization

Training instability in margin-based angular losses primarily arises from complex or oscillatory margin functions with nontrivial derivatives. The “characteristic gradient detachment” (CGD) method ensures stable training by detaching the margin function $\Delta(\theta)$ from backpropagation in SphereFace-R, making the angular gradient resemble that of the basic normalized softmax loss.

Feature normalization schemes impact the learned representation:
- **No feature normalization (NFN):** $\|\mathbf{x}\|$ unconstrained.
- **Hard feature normalization (HFN):** $\|\mathbf{x}\|=s$ enforced.
- **Soft feature normalization (SFN):** Penalty regularizes $\|\mathbf{x}\|$ toward $s$, retaining magnitude information.

Empirical results confirm that applying CGD and SFN with SphereFace-R eliminates oscillating loss trajectories and enables convergence matching the best additive-margin schemes [2109.05565].

## 7. Empirical Performance and Practical Impact

Experiments across VGGFace2, MS-Celeb-1M, MegaFace, and IJB benchmarks demonstrate the following trends:
- SphereFace with proper normalization and CGD is competitive or superior in low false-accept-rate (FAR) regimes, especially with soft normalization.
- ArcFace and CosFace maintain robust performance with high training stability [2109.05565].

Overall, SphereFace, CosFace, and ArcFace formalize the margin-based softmax landscape for hyperspherical face recognition, each balancing geometric margin strength, focus of penalty, and optimization stability within a unified framework [2509.19753][2109.05565].

Source: https://www.emergentmind.com/topics/sphereface-cosface-and-arcface