---
title: 'CosFace Loss: Additive Cosine Margin'
url: https://www.emergentmind.com/topics/cosface-loss
type: topic
---

# CosFace Loss: Additive Cosine Margin

Searching arXiv for recent and foundational papers on CosFace and closely related variants.
CosFace Loss, introduced in “CosFace: Large Margin Cosine Loss for Deep Face Recognition” as **Large Margin Cosine Loss (LMCL)**, is a normalized softmax-based classification objective in which both features and classifier weights are constrained to a hypersphere and the target-class cosine logit is reduced by a fixed additive margin. The design aligns training with cosine-similarity-based verification and identification, which is the dominant inference geometry in modern face recognition [1801.09414]. In later comparative literature, the same method is frequently discussed under the name **Additive-Margin Softmax Loss (AM-Softmax)** [1901.05903].

## 1. Formal definition and nomenclature

The original formulation starts from the standard softmax classifier, with class logit
\[
f_j = W_j^T x_i + b_j,
\]
and cross-entropy loss
\[
L_s = -\frac{1}{N}\sum_{i=1}^N \log \frac{e^{f_{y_i}}}{\sum_{j=1}^C e^{f_j}}.
\]
CosFace rewrites this classifier in normalized angular form by imposing
\[
\|W_j\| = 1,\qquad b_j = 0,
\]
normalizing the feature, and rescaling it by a fixed constant \(s\). The normalized logit becomes
\[
f_j = s\cos\theta_{j,i},
\]
where \(\theta_{j,i}\) is the angle between the sample feature and class weight. CosFace then modifies only the target-class logit:
\[
f_{y_i} = s(\cos\theta_{y_i,i}-m),\qquad f_j = s\cos\theta_{j,i}\;\;(j\neq y_i).
\]
The resulting loss is
\[
L_{\mathrm{LMCL}}=
-\frac{1}{N}\sum_{i=1}^N
\log
\frac{e^{s(\cos\theta_{y_i,i}-m)}}
{e^{s(\cos\theta_{y_i,i}-m)}+\sum_{j\neq y_i}e^{s\cos\theta_{j,i}}}.
\]
Here \(m\ge 0\) is the additive cosine margin and \(s\) is the scale factor [1801.09414].

Comparative work consistently identifies this same objective as AM-Softmax. In that naming, CosFace is the case in which the target logit is replaced by \(\cos\theta_{y_i}-m\) while all non-target logits remain \(\cos\theta_j\), and all cosine logits are scaled by \(s\) before softmax [1901.05903]. The two names therefore refer to the same normalized additive-cosine-margin construction.

## 2. Hyperspherical geometry and decision boundaries

The geometric premise of CosFace is that deep face recognition should be governed by angular relations rather than Euclidean norm magnitude. By \(L_2\)-normalizing both features and class weights, CosFace removes radial variation and constrains optimization to a hypersphere. This is important because test-time matching is typically based on cosine similarity between embeddings rather than raw inner products [1801.09414].

Under normalized softmax, the binary decision boundary is
\[
\cos\theta_1=\cos\theta_2.
\]
CosFace makes this boundary stricter. For two classes \(C_1\) and \(C_2\), the class-conditional requirements become
\[
C_1:\ \cos\theta_1 \ge \cos\theta_2 + m,
\qquad
C_2:\ \cos\theta_2 \ge \cos\theta_1 + m.
\]
Equivalently, the target cosine must exceed the competing cosine by at least \(m\). This directly enforces tighter intra-class angular concentration and larger inter-class separation on the hypersphere [1801.09414].

Later work sharpened the interpretation of this boundary. GB-CosFace characterized ordinary CosFace as a **relative inter-class boundary**: training pushes the target score above competing class scores, especially the largest non-target score, whereas open-set verification at test time compares two embeddings against a **global threshold**. In that view, CosFace remains highly effective, but it still inherits the closed-set multiclass structure of softmax training [2111.11186].

The same hyperspherical interpretation also explains why CosFace transfers naturally beyond canonical face verification. In VMCML, for example, CosFace is used as the “cross-modality lifting loss” that places video and music embeddings in a shared hyperspherical space, again because cosine similarity is the downstream retrieval criterion [2303.12379].

## 3. Scale, margin, and optimization behavior

CosFace has two central hyperparameters: the scale \(s\) and the additive cosine margin \(m\). The scale is required because cosine values lie in a bounded interval, so normalized logits are otherwise too small for effective softmax optimization. The original paper derives a lower bound for \(s\),
\[
s \geq \frac{C-1}{C}\log\frac{(C-1)P_W}{1-P_W},
\]
where \(P_W\) is the expected minimum posterior probability at a class center. In experiments, the paper fixes
\[
s=64
\]
and reports that performance improves as \(m\) increases, saturates around
\[
m=0.35,
\]
and fails to converge when
\[
m>0.45
\]
in its setup [1801.09414].

The same sensitivity was later analyzed in detail by AdaCos. That work argues that both \(s\) and \(m\) modulate the mapping from cosine similarity to predicted class probability: too small a scale prevents probabilities from becoming confident enough, while too large a scale can make the loss insensitive; too small a margin weakens supervision, and too large a margin can make convergence difficult [1905.00292]. AdaCos therefore treats CosFace as a strong but hyperparameter-sensitive member of the cosine-softmax family.

Comparative studies support this interpretation. The 2019 benchmark paper on face-recognition losses reports that AM-Softmax is consistently among the strongest losses, usually second only to ArcFace and occasionally best in a specific setting, but it does not provide the exact CosFace-specific \(m\) and \(s\) values used in its own experiments [1901.05903]. In other words, CosFace is both simple and effective, but its behavior remains materially dependent on scale-margin calibration.

## 4. Position within the margin-softmax family

CosFace sits in the core lineage of hyperspherical margin losses for face recognition. The principal distinctions are in **where** the margin is injected.

| Loss | Target transformation | Paper characterization |
|---|---|---|
| Softmax | \(\cos\theta\) | No explicit margin |
| SphereFace | \(\cos(m\theta)\) | Multiplicative angular margin |
| CosFace | \(\cos\theta-m\) | Additive cosine margin |
| ArcFace | \(\cos(\theta+m)\) | Additive angular margin |

This progression is presented explicitly in comparative work and in later papers that extend the family [1901.05903].

Several subsequent losses reinterpret or generalize CosFace rather than discard it. GB-CosFace introduces an adaptive global boundary \(p_v\) and proves that CosFace is a special case obtained at \(\alpha=0\); in its rewritten form, that special case is equivalent to CosFace with margin \(2m\) and scale \(s\) [2111.11186]. ElasticFace-Cos replaces CosFace’s fixed additive margin with a Gaussian-sampled random margin \(E(m,\sigma)\), and reduces exactly to CosFace when \(\sigma=0\) [2109.09416]. Q-Margin moves the margin from direct logit modification into the reference measure of an \(\alpha\)-divergence loss and recovers CosFace in the limit \(\alpha\to 1\) [2606.31664].

Other work critiques CosFace from a more geometric or adaptive-margin perspective. ExpFace treats CosFace as a uniform downward shift of the target similarity curve, emphasizing that its penalty is constant in similarity space and that the maximum of its gradient curve remains at \(\pi/2\) as the margin changes [2509.19753]. X2-Softmax argues that fixed-margin methods such as CosFace are mismatched to imbalanced class geometry and proposes an adaptive angular margin based on a quadratic target-logit function [2312.05281]. Across these developments, CosFace remains the reference additive-cosine baseline against which newer geometric, probabilistic, or adaptive constructions are defined.

## 5. Adaptations, hybrids, and non-face deployments

Although introduced for deep face recognition, CosFace has been adapted in several distinct ways without altering its core additive-cosine-margin structure. MultiFace applies CosFace independently to multiple low-dimensional sub-features and sums the branch losses; it reports that MultiFace accelerates training by about \(1.2\)–\(1.5\times\) with ArcFace or CosFace while improving benchmarks such as MegaFace and IJB [2101.09899]. SubFace instead approximates full-feature training by randomly selected normalized subspaces; in its unified implementation, CosFace corresponds to the additive-cosine case \(\phi = cosine - m\), although the paper concludes that ArcFace generally outperforms CosFace under the same subspace strategy [2208.11483].

CosFace has also been transplanted beyond face recognition. VMCML uses a shared prototype matrix for both video and music embeddings and defines a “cross-modality lifting loss”
\[
L_{LL}(g(v),f(m),W)=L_C(g(v),W)+\alpha L_C(f(m),W),
\]
thereby adapting CosFace from face identities to video–music correspondence classes in a shared hyperspherical space [2303.12379]. In text-independent speaker identification, a modified VGG16 with \(256\)-dimensional \(L_2\)-normalized embeddings is trained with CosFace using \(s=22\) and \(m=0.2\), and the paper reports the best result of \(83.15\%\) top-1 accuracy on VoxCeleb1, ahead of ArcFace and Softmax in that setup [2509.22838].

Within face-recognition systems, CosFace has also served as a modular building block rather than the final method. IronMask uses CosFace embeddings as normalized angular templates inside a protection architecture based on orthogonal transforms and real-valued ECC; the paper reports that CosFace is compatible with this protection scheme, but suffers larger degradation than ArcFace unless a centering step is added [2104.02239]. UniTSFace combines CosFace with a unified-threshold integrated sample-to-sample loss, reporting clear gains over CosFace alone on MR-All and IJB-C while keeping the sample-to-class component [2311.02523].

## 6. Empirical profile, limitations, and continuing development

The original CosFace paper reports that feature normalization is crucial and that LMCL outperforms softmax, center loss, triplet loss, and A-Softmax under matched settings. On CASIA-WebFace with normalized features, the paper reports LFW \(99.33\), YTF \(96.1\), MegaFace1 Rank-1 \(77.11\), and MegaFace1 Verification \(89.88\); with a larger training set, it reports LFW \(99.73\) and YTF \(97.6\) [1801.09414]. These results established CosFace as a major practical alternative to earlier angular-margin losses.

Later comparisons give a more nuanced empirical picture. The 2019 loss-comparison study ranks ArcFace first overall and Additive-Margin Softmax second overall, but also reports a setting in which AM-Softmax is best: ResNet50 trained on MS-Celeb-1M reaches \(99.30\%\) on LFW, slightly ahead of ArcFace in that particular experiment [1901.05903]. Subsequent methods often improve over CosFace at stricter verification operating points: GB-CosFace reports gains on IJB-B and IJB-C, especially at very low FAR, and Q-Margin reports controlled improvements over CosFace and ArcFace baselines on IJB-B, IJB-C, and VoxCeleb in identical training recipes [2111.11186] [2606.31664].

The principal limitations attributed to CosFace in later literature are consistent. First, its **fixed margin** may be unrealistic when intra-class and inter-class variation are highly heterogeneous; this is the central motivation of ElasticFace and X2-Softmax [2109.09416] [2312.05281]. Second, its closed-set multiclass training objective does not perfectly match open-set verification, where decisions are made using a global threshold on similarity rather than relative ranking among training classes [2111.11186]. Third, its performance is materially influenced by manual tuning of \(s\) and \(m\), a point emphasized by AdaCos [1905.00292]. Fourth, several papers argue that generic margin losses, including CosFace, are less effective on hard samples than on easy ones, motivating distribution-level or quality-aware modifications such as DDL and LH\(^2\)Face [2002.03662] [2506.23555].

Taken together, these developments place CosFace in a distinctive position. It is the canonical additive-cosine-margin loss: simple, hyperspherical, easy to implement, and empirically strong across face recognition, speaker recognition, and even cross-modal retrieval. At the same time, much of the subsequent literature can be read as an extended attempt to preserve CosFace’s normalized cosine geometry while correcting its fixed-margin, fixed-scale, or closed-set limitations.

Source: https://www.emergentmind.com/topics/cosface-loss