---
title: 'ExpFace: Exponential Angular Margin Loss'
url: https://www.emergentmind.com/topics/exponential-angular-margin-loss-expface
type: topic
---

# ExpFace: Exponential Angular Margin Loss

Searching arXiv for ExpFace and closely related angular-margin face-recognition losses.
Exponential Angular Margin Loss (ExpFace) is a margin-based face-recognition loss that replaces the target-class logit of normalized softmax with an exponential angular transform, \(T_{ExpFace}(\theta)=\cos\!\left(\pi\left(\frac{\theta}{\pi}\right)^{m_e}\right)\). It was introduced for deep face recognition as a response to a specific robustness problem in large-scale open-set training: standard angular-margin losses such as SphereFace, CosFace, and ArcFace improve discrimination, but the ExpFace paper argues that they do not explicitly account for the impact of noisy samples. Its central design principle is to apply a larger penalty in the center region of angular space and a smaller penalty in the peripheral region, thereby emphasizing clean samples while suppressing noisy samples [2509.19753].

## 1. Genealogy within angular-margin face recognition

ExpFace belongs to the family of normalized margin-based softmax objectives that learn embeddings on a hypersphere by modifying the target-class logit as a function of the angle between a feature and its class weight. This lineage begins with early large-margin softmax formulations that explicitly altered the angular decision geometry of the classifier; in particular, L-Softmax established the general principle that feature discrimination can be improved by replacing the target-class \(\cos(\theta_y)\) with a stricter angular transform [1612.02295]. ExpFace inherits this hyperspherical viewpoint, but changes the shape of the angular margin itself rather than using the multiplicative, additive-cosine, or additive-angular constructions of earlier losses [2509.19753].

A concise comparison of the target-logit forms discussed around ExpFace is given below.

| Method | Target-logit form | Margin type |
|---|---|---|
| Softmax | \(T(\theta)=\cos\theta\) | none |
| SphereFace | \(T(\theta)=(-1)^k\cos(m_s\theta)-2k\) | multiplicative angular |
| ArcFace | \(T(\theta)=\cos(\theta+m_a)\) | additive angular |
| ExpFace | \(T(\theta)=\cos\!\left(\pi\left(\frac{\theta}{\pi}\right)^{m_e}\right)\) | exponential angular |

Within this progression, ExpFace is not a prototype-based reweighting method, a support-vector-guided denominator modification, or a class-adaptive additive margin. Its contribution is specifically a new angular embedding form for the target class. The paper positions this embedding form as better aligned with the empirical angular distribution of clean and noisy training samples than the classical fixed-margin alternatives [2509.19753].

## 2. Mathematical formulation

The starting point is the standard normalized softmax setting. Let \(x_i \in \mathbb{R}^d\) be the feature of sample \(i\), \(y_i\) its class label, and \(W_j\) the class weight for class \(j\). After \(L_2\)-normalizing features and class weights, removing biases, and introducing a scale factor \(s\), the loss becomes
\[
L= -\frac{1}{N}\sum^{N}_{i=1}  \ln\frac{e^{sT(\theta_{y_{i}i})}}{e^{sT(\theta_{y_{i}i})}+\sum^{C}_{j=1;j\ne y_{i}}e^{s\cos{\theta_{ji}}}},
\]
where \(\theta_{ji}\) is the angle between \(x_i\) and \(W_j\), and vanilla softmax uses
\[
T(\theta_{y_{i}i})=\cos{\theta_{y_{i}i}}.
\]
ExpFace modifies only this target-logit term; non-target logits remain \(s\cos\theta_{ji}\) [2509.19753].

The paper first proposes a direct exponential-angular form,
\[
T_{ExpFace}(\theta_{y_{i}i})=\cos{(\theta_{y_{i}i}^{m_e})},
\]
but then states that this direct version fails to penalize large-angle samples effectively over the full range \([0,\pi]\). Its final construction is a shrink-then-expand transform:
\[
T_{ExpFace}(\theta_{y_{i}i})=\cos{\left(\pi\left(\frac{\theta_{y_{i}i}}{\pi}\right)^{m_e}\right)}.
\]
Equivalently, defining
\[
\phi(\theta)=\pi\left(\frac{\theta}{\pi}\right)^{m_e},
\]
the target logit is \(T_{ExpFace}(\theta)=\cos(\phi(\theta))\). Substituting this into normalized softmax yields
\[
L_{ExpFace} = -\frac{1}{N}\sum^{N}_{i=1}  \ln \frac{e^{s\cos\left(\pi\left(\frac{\theta_{y_{i}i}}{\pi}\right)^{m_e}\right)}} {e^{s\cos\left(\pi\left(\frac{\theta_{y_{i}i}}{\pi}\right)^{m_e}\right)}+\sum^{C}_{j=1;j\ne y_{i}}e^{s\cos{\theta_{ji}}}}.
\]

The practical setting reported in the experiments uses \(m_e=0.7\) and \(s=64\). Since \(0<m_e<1\), for normalized angle \(u=\theta/\pi \in (0,1)\), one has \(u^{m_e}>u\), so \(\phi(\theta)>\theta\) and therefore \(\cos(\phi(\theta))<\cos(\theta)\). This yields a margin penalty, but not a uniform one: the magnitude of the angular increase varies across \(\theta\), which is the defining mechanism of ExpFace [2509.19753].

## 3. Angular-space interpretation and the noise-robustness hypothesis

The defining claim of ExpFace is not merely that a nonlinear angular transform increases class separation, but that its particular nonlinearity matches the empirical placement of clean and noisy samples in angular space. The paper analyzes a 2D angular space defined by the angles from a sample to its positive class center and to competing negative class centers, and reports three regions of interest. Clean samples mainly cluster in the center region. Type-I noisy samples—samples whose true identity matches neither the assigned positive class nor any negative class—tend to drift toward the upper-right boundary region, where both positive and negative angles are large. Type-II noisy samples—samples from one training identity that are incorrectly split across multiple classes—tend to drift toward the lower-left boundary region, where both positive and some negative angles are small [2509.19753].

This distribution motivates a nonuniform margin. If clean samples inhabit the center region while noisy samples accumulate near peripheral or boundary zones, then a loss that penalizes the center more strongly and the periphery more weakly should concentrate optimization on cleaner supervision. ExpFace operationalizes that intuition by making the effective transformed-angle increment
\[
\Delta(\theta)=\phi(\theta)-\theta=\pi\left(\frac{\theta}{\pi}\right)^{m_e}-\theta
\]
largest in the intermediate angular region and smaller near \(\theta\approx 0\) and \(\theta\approx \pi\). Since the target logit is \(\cos(\phi(\theta))\), larger \(\Delta(\theta)\) corresponds to a stronger target penalty. The result is a center-heavy margin distribution rather than a uniform or boundary-heavy one [2509.19753].

The paper contrasts this behavior with earlier losses in qualitative angular-space terms. SphereFace is described as emphasizing samples moving from lower-left to upper-right along the diagonal; CosFace as increasing margin from the center toward the boundary; ArcFace as being mostly uniform, with slightly larger effects near the endpoints; and ExpFace as increasing margin from boundary toward center. On that basis, the authors argue that ExpFace better matches the desired clean-sample emphasis under residual label noise [2509.19753].

## 4. Similarity curves, gradient behavior, and stability claims

The ExpFace paper compares classical angular-margin losses through margin embedding forms, similarity curves, and gradient curves. Its argument is that the geometry of the target-logit curve \(T(\theta)\) and the gradient \(\partial L/\partial \theta\) jointly determine not only discrimination but also the angular region that receives the strongest optimization pressure [2509.19753].

For analysis, the paper approximates all negative-class angles by an average angle \(b\), following AdaCos in taking \(b\approx \pi/2\), and rewrites the one-sample loss as
\[
L=\ln{\left(1+e^{-sT(\theta_{y_{i}i})+s\cos{(b)}+\ln(C-1)}\right)}.
\]
It defines a transition angle \(\theta_{trans}\) by setting the target posterior to \(1/2\), which yields
\[
\theta_{trans}=T^{-1}\left(\cos b+\frac{\ln{(C-1)}}{s}\right).
\]
For the vanilla case \(T(\theta)=\cos\theta\), this becomes
\[
\theta_{trans}=\arccos\left(\cos b+\frac{\ln(C-1)}{s}\right).
\]
Margin-based losses move \(\theta_{trans}\) leftward, making the classification criterion stricter [2509.19753].

For ExpFace,
\[
T_{ExpFace}(\theta)=\cos\left(\pi\left(\frac{\theta}{\pi}\right)^{m_e}\right),
\]
and the paper gives
\[
T'_{ExpFace}(\theta) = -\,m_e\left(\frac{\theta}{\pi}\right)^{m_e-1} \sin\left(\pi\left(\frac{\theta}{\pi}\right)^{m_e}\right),
\]
with the corresponding angle-gradient magnitude
\[
\left|\frac{\partial L}{\partial \theta}\right| = (1-P_i)\,s\,m_e\left(\frac{\theta}{\pi}\right)^{m_e-1} \sin\left(\pi\left(\frac{\theta}{\pi}\right)^{m_e}\right).
\]
The paper’s qualitative conclusions are that SphereFace develops multiple peaks and troughs as its multiplicative margin grows, which is associated with training instability; CosFace keeps its maximum gradient at \(\pi/2\), so changing its margin mainly shifts the convergence criterion rather than the angular focus; ArcFace develops a negative-gradient region as the margin grows, implying that some samples would be pushed to increase their target angle; and ExpFace avoids SphereFace-style oscillatory growth and ArcFace’s negative-gradient region while allowing the location and maximum of the gradient to vary regularly with \(m_e\) [2509.19753].

A later paper on angular-margin design argues more generally that explicit angle recovery via \(\arccos\) can introduce boundary-gradient explosion in AAM-Softmax-style formulations [2601.13198]. This suggests that ExpFace should also be examined through the broader lens of optimization geometry, although the ExpFace paper itself frames its stability claims primarily in terms of similarity-curve monotonicity and gradient-shape comparisons with SphereFace and ArcFace rather than through an explicit \(\arccos\)-based instability analysis [2601.13198].

## 5. Training protocol and empirical performance

ExpFace is presented as a drop-in replacement for the target-logit transformation in a standard margin-softmax face-recognition pipeline. The reported implementation uses a ResNet50 backbone, a 512-dimensional embedding, normalized \(112\times112\) face crops aligned with five facial landmarks, PyTorch with apex mixed precision, SGD with initial learning rate \(0.02\), momentum \(0.9\), weight decay \(5\times10^{-4}\), batch size \(256\), and training for 20 epochs on a single GPU. The training datasets are CASIA-WebFace, MS1MV3, and WebFace4M. Baseline settings are SphereFace \([s=32,\; m=1.7]\), CosFace \([s=64,\; m=0.4]\), ArcFace \([s=64,\; m=0.5]\), and ExpFace \([s=64,\; m_e=0.7]\) [2509.19753].

On six standard verification benchmarks, ExpFace is reported as competitive and often best on harder sets. With CASIA training, it achieves LFW \(98.783\), CPLFW \(86.517\), CALFW \(92.767\), CFP-FP \(93.057\), AgeDB-30 \(92.383\), and VGG2-FP \(89.4\), with the paper marking the latter five as best among the compared losses. With MS1MV3 training, it attains CALFW \(96.233\) as best and remains second-best on CPLFW and VGG2-FP. With WebFace4M training, it reaches CPLFW \(94.25\) as tied best, CFP-FP \(99.129\) as best, and AgeDB-30 \(97.8\) as best [2509.19753].

The paper emphasizes low-FAR performance on IJB-B and IJB-C. For CASIA-trained models, ExpFace reports IJB-B TAR@\(10^{-4}\) of \(77.62\) and IJB-C TAR@\(10^{-4}\) of \(82.08\), exceeding the reported SphereFace, CosFace, and ArcFace baselines under the same setup. With MS1MV3 training, it reaches IJB-B TAR@\(10^{-4}\) of \(95.37\) and IJB-C TAR@\(10^{-4}\) of \(96.69\), both reported as best. With WebFace4M training, it attains IJB-B TAR@\(10^{-4}\) of \(95.23\) as best and IJB-C TAR@\(10^{-4}\) of \(96.91\), just behind ArcFace’s \(96.95\). On MegaFace, ExpFace is again strong: CASIA-trained identification \(83.379\) as best; MS1MV3-trained verification \(98.538\), second only to SphereFace’s \(98.595\); and WebFace4M-trained verification \(97.896\), second to ArcFace’s \(98.001\) [2509.19753].

The paper also reports that ExpFace benefits particularly from a dynamic margin strategy, although the exact schedule is not specified in the provided text. Under that strategy, ExpFace is said to improve by about \(+3.6\%\) on IJB-B and about \(+3.1\%\) on IJB-C, while classical losses fluctuate more. The stated practical guidance is that \(m_e\) should not be too small or too large because that may cause gradient explosion; values in \([0.3,10]\) are described as sufficient for most applications without causing gradient issues, and \(m_e=0.7\) is the setting used throughout the main experiments [2509.19753].

## 6. Distinctions, common confusions, and limitations

ExpFace is sometimes liable to be confused with other exponential or adaptive angular losses, but the relevant formulations are materially different. “Scalable Angular Discriminative Deep Metric Learning” does not introduce a method named ExpFace; its closest related construction is DLMC, which compares intra-class and inter-class cosine similarities in a normalized exponential feature projection space, but it remains a separate regularizer attached to normalized softmax rather than the exponential angular target-logit transform of ExpFace [1804.10899]. “Bi-directional Exponential Angular Triplet Loss” is also different: it is a triplet-based ranking objective for RGB-Infrared person re-identification, acting on sample-to-sample cosine similarities rather than on class-prototype logits in face-recognition softmax [2006.00878].

ExpFace is equally distinct from adaptive ArcFace-style methods. KappaFace is a class-adaptive additive angular margin loss in which the ArcFace margin is linearly rescaled according to class population imbalance and vMF concentration; it is explicitly not an exponential-margin method, and its adaptation is classwise rather than based on the center-versus-periphery hypothesis that defines ExpFace [2201.07394]. Support Vector Guided Softmax Loss follows another direction again: it emphasizes boundary-violating non-target classes through support-vector-guided denominator reweighting, which the paper presents as complementary to margin-based target-logit design rather than a substitute for it [1812.11317].

A broader theoretical lens comes from loss-search work that argues the core effect of margin-based softmax losses is how they reduce target softmax probability. This suggests that ExpFace can plausibly also be interpreted through posterior suppression induced by its transformed target logit, although that is an inference rather than the explicit framing of the ExpFace paper itself [2007.06542]. By contrast, the ExpFace paper’s own explanation remains geometric and empirical: clean samples are assumed to dominate the center of angular space, noisy samples are assumed to drift toward peripheral regions, and the exponential transform is chosen so that the similarity curve and gradient curve emphasize the former and suppress the latter [2509.19753].

The principal limitations stated or implied in the ExpFace paper are correspondingly specific. Its gains over ArcFace and CosFace are often incremental rather than dramatic on larger and relatively clean datasets. Its robustness argument depends on the empirical assumption that clean and noisy samples occupy different angular regions in the reported way. The paper does not provide a fully explicit probabilistic noise model or a formal theorem of robustness. Parameter choice also matters: extreme \(m_e\) values may cause gradient explosion. A plausible implication is that ExpFace is best understood not as a universally dominant replacement for classical margin losses, but as a particular re-shaping of angular supervision designed for regimes where residual label noise and low-FAR verification performance are central concerns [2509.19753].

Source: https://www.emergentmind.com/topics/exponential-angular-margin-loss-expface