DistArc Loss: Radial-Angular Embedding

• DistArc Loss is a loss function that combines radial and angular components, arranging embeddings on multiple concentric hyperspheres or augmenting ArcFace with distribution distillation.
• In the HyperSpaceX framework, per-class radii and dual angular terms address angular crowding by separating class clusters both by direction and distance.
• In face recognition applications, the ArcFace model augmented with Distribution Distillation Loss narrows the performance gap between easy and hard samples through distribution alignment.

Searching arXiv for the papers and terminology around “DistArc Loss” to ground the article in the cited literature. DistArc Loss denotes two distinct constructs in recent arXiv literature. In the 2024 HyperSpaceX framework, “DistArc” is the official name of a radial–angular classification loss that combines two angular components and one radial component to arrange embeddings on multiple concentric hyperspheres (Chiranjeev et al., 2024). In a separate 2020 face-recognition context, “DistArc” is not an official term but a convenient shorthand for ArcFace classification loss augmented with Distribution Distillation Loss (DDL), yielding an objective of the form $L_{\text{ArcFace}} + L_{\text{DDL}}$ (Huang et al., 2020). The shared label therefore masks a substantive distinction: one formulation is a native radial–angular loss for multi-hyperspherical representation learning, whereas the other is an ArcFace instantiation of distribution-level distillation from easy to hard samples.

1. Terminological scope and literature usage

The term “DistArc” is used officially in HyperSpaceX, where it is presented as the centerpiece of a framework for “radial and angular exploration of HyperSpherical Dimensions” (Chiranjeev et al., 2024). By contrast, in the face-recognition paper on Distribution Distillation Loss, “DistArc” does not appear in the paper; it is only a convenient shorthand for “ArcFace + DDL,” meaning an ArcFace-based classifier/backbone regularized by DDL (Huang et al., 2020).

Usage	Source	Meaning
DistArc	(Chiranjeev et al., 2024)	Official radial–angular loss in HyperSpaceX
“DistArc” shorthand	(Huang et al., 2020)	ArcFace classification loss plus DDL regularization
Distributional adversarial loss	(Ahmadi et al., 2024)	Distinct adversarial-risk notion over perturbation distributions

This distinction is not merely terminological. In HyperSpaceX, DistArc is designed to alleviate angular crowding by distributing classes across radii as well as directions. In the 2020 face-recognition setting, the ArcFace+DDL combination instead distills the similarity-distribution structure of easy samples into hard samples. A plausible implication is that the same informal name can refer either to a geometry-aware embedding loss or to a distillation-regularized ArcFace objective, depending on context.

2. HyperSpaceX DistArc: radial–angular formulation

In HyperSpaceX, DistArc operates in a multi-hyperspherical embedding space in which classes are organized not only by direction but also by distance from the origin (Chiranjeev et al., 2024). For sample $i$ with label $c := y_i$, the feature is $x_i \in \mathbb{R}^d$, and each class $j$ has proxy $W_j \in \mathbb{R}^d$ with normalized direction

$\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$

Angular similarity is defined by

$$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$

The distinctive element is the introduction of per-class radii $r_j > 0$, which place class proxies on concentric hyperspheres: $\omega_{r_j} = r_j \hat{W}_j.$ For the true class $i$0, the resultant vector is

$i$1

and DistArc defines a second angle $i$2 through

$i$3

The paper describes this geometrically as pulling the tip of $i$4 toward the tip of its scaled proxy on the appropriate hypersphere.

Radial structure enters through squared distances to scaled proxies: $i$5 DistArc then combines three components in the logits: a primary angular component with additive angular margin $i$6, a secondary angular component $i$7, and a radial penalty weighted by $i$8. A faithful parameterization given in the paper details is

$i$9

$c := y_i$0

with final cross-entropy

$c := y_i$1

The stated roles of the three components are sharply differentiated. $c := y_i$2 enforces angular separation around class directions; $c := y_i$3 compacts the class in both angle and radius by aligning the feature with the class’s scaled-proxy tip; and $c := y_i$4 binds features to their own radius while discouraging proximity to other classes’ radii. This suggests that HyperSpaceX treats class discrimination as a joint problem of angular margin and radial assignment rather than a purely hyperspherical angular packing problem.

3. Geometry, normalization policy, and predictive rule

The HyperSpaceX paper is explicit that angular and radial terms operate under different normalization policies (Chiranjeev et al., 2024). Angular components use L2-normalized features and proxies to compute $c := y_i$5, while radial components use the raw feature magnitude $c := y_i$6. Proxies are first L2-normalized and then scaled by per-class radii to form $c := y_i$7. The radii $c := y_i$8 are treated as per-class quantities that may be fixed, scheduled, or made learnable with positivity constraints; the empirical results rely on non-uniform per-class radii to realize multi-hyperspherical separation.

The intended geometry is a set of concentric hyperspheres centered at the origin. The first angular term separates class clusters by direction on their assigned spheres. The second angular term evaluates whether a feature points toward the tip of the class’s scaled proxy at radius $c := y_i$9. The radial term constrains occupancy of radial bands. The paper’s qualitative interpretation is that this relieves angular competition when many classes would otherwise need to share a single unit sphere.

At inference, HyperSpaceX does not use standard cosine-only scoring. Instead, it introduces a predictive measure based on proximity to scaled proxies: $x_i \in \mathbb{R}^d$0 The paper also gives a law-of-cosines-style expression,

$x_i \in \mathbb{R}^d$1

while noting that direct computation of $x_i \in \mathbb{R}^d$2 is simplest and numerically stable.

This inference rule is central to the method’s identity. It means that DistArc is not merely a training-time regularizer on top of a conventional angular classifier; it changes the representation geometry and the decision rule simultaneously. A plausible implication is that its benefits are expected to be most visible when angular-only embeddings are capacity-limited, such as low-dimensional regimes or large-class settings.

4. HyperSpaceX training procedure, hyperparameters, and empirical profile

The training procedure specified for DistArc follows a standard backbone-plus-proxy pipeline with additional radial computations (Chiranjeev et al., 2024). For each batch, one computes feature vectors $x_i \in \mathbb{R}^d$3, normalized directions $x_i \in \mathbb{R}^d$4 and $x_i \in \mathbb{R}^d$5, scaled proxies $x_i \in \mathbb{R}^d$6, angular similarities $x_i \in \mathbb{R}^d$7, the true-class margin term $x_i \in \mathbb{R}^d$8, the resultant vector $x_i \in \mathbb{R}^d$9, the second angle $j$0, and radial distances $j$1. The resulting logits are passed through softmax and optimized by backpropagation through backbone parameters and proxies; if $j$2 are learnable, they are updated with positivity constraints.

The reported hyperparameters include a typical angular margin $j$3, radial weight $j$4 for object classification depending on dataset complexity, and for face recognition a schedule from $j$5 to $j$6 incremented by $j$7 every ten epochs and then held fixed. Backbones include iResNet50, RN101 (CLIP), ViT-B, and ViT-L. For simple datasets such as MNIST and FashionMNIST, the paper uses SGD with $j$8 and weight decay $j$9; for face recognition on CASIA-WebFace, it uses SGD with $W_j \in \mathbb{R}^d$0 and weight decay $W_j \in \mathbb{R}^d$1. Embedding sizes span 2D, 512D, and 2048D.

The empirical profile emphasizes gains in low-dimensional and high-class-count settings. The paper reports, among other results, the following improvements over softmax or angular-loss baselines: TinyImageNet gains of $W_j \in \mathbb{R}^d$2 in 2D, $W_j \in \mathbb{R}^d$3 in 512D, and $W_j \in \mathbb{R}^d$4 in 2048D; CIFAR-100 gains of $W_j \in \mathbb{R}^d$5 in 2D, $W_j \in \mathbb{R}^d$6 in 512D, and $W_j \in \mathbb{R}^d$7 in 2048D; and for CUB-200 with ViT-L at 2D, an improvement of $W_j \in \mathbb{R}^d$8 (Chiranjeev et al., 2024). On ImageNet-1K with iResNet50, DistArc leads at 32D and 128D and is second-best at 512D. For face recognition with iResNet50 and 512D embeddings, the paper reports LFW $W_j \in \mathbb{R}^d$9, CFP-FP $\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$0, AgeDB-30 $\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$1, CA-LFW $\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$2, and CP-LFW $\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$3, with MS1Mv2 results including LFW $\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$4 and AgeDB-30 $\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$5.

Ablations attribute improvements to all three components. Using only $\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$6 is the weakest setting; adding $\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$7 helps; adding radial $\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$8 helps more; and combining $\hat{W}_j = \frac{W_j}{\|W_j\|_2}.$9 yields the best performance across MNIST, FashionMNIST, and CIFAR-10. The paper further notes that relative to ArcFace, DistArc adds per-class radial distances $$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$0 and one extra angle $$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$1 per sample, with $$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$2 cost comparable to logits and minor overhead under vectorized implementations.

5. ArcFace plus Distribution Distillation Loss: the alternate “DistArc” usage

In the 2020 face-recognition literature, the relevant formal object is Distribution Distillation Loss (DDL), proposed to improve performance on hard samples by narrowing the performance gap between easy and hard samples (Huang et al., 2020). Under the shorthand mapping given in the provided material, “DistArc” means DDL instantiated on top of an ArcFace classifier/backbone: ArcFace provides the classification loss, while DDL regularizes pairwise similarity distributions.

ArcFace is defined on normalized embeddings $$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$3 and normalized class weights $$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$4, with $$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$5. Its logits are

$$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$6

and the classification term is the cross-entropy over the softmax of these logits.

DDL acts not on class-wise logits but on pairwise cosine similarities in the embedding space. For each subset—teacher and student—it estimates two one-dimensional distributions: a positive-pair similarity distribution for same-identity pairs and a negative-pair similarity distribution for different-identity pairs. Teacher denotes easy samples and student denotes hard samples. The split is task-specific and fixed before training: on SCface, HR ($$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$7) are easy and LR ($$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$8) are hard; on VGGFace2-based pose experiments, easy means yaw $$\cos\theta_{ij} = \frac{x_i^\top W_j}{\|x_i\|_2\,\|W_j\|_2} = \hat{x}_i^\top \hat{W}_j, \quad \text{where } \hat{x}_i = \frac{x_i}{\|x_i\|_2}.$$9 and hard means yaw $r_j > 0$0; on COX, Caucasian faces are easy and Mongolian faces hard because the CASIA pre-training set is biased to Caucasian (Huang et al., 2020).

Within each mini-batch, positive pair similarities are

$r_j > 0$1

and negative pair similarities are obtained by online hard mining: $r_j > 0$2 where identities differ. Positive pairs with $r_j > 0$3 are treated as outliers and removed.

Soft histograms over cosine similarities in $r_j > 0$4 are built using uniformly spaced nodes $r_j > 0$5, step $r_j > 0$6, and Gaussian soft assignment

$r_j > 0$7

These produce normalized teacher histograms $r_j > 0$8 and student histograms $r_j > 0$9. DDL then combines a KL term

$\omega_{r_j} = r_j \hat{W}_j.$0

where

$\omega_{r_j} = r_j \hat{W}_j.$1

with an order loss

$\omega_{r_j} = r_j \hat{W}_j.$2

The total loss is

$\omega_{r_j} = r_j \hat{W}_j.$3

The stated mechanism is distributional rather than samplewise. KL divergence forces the student’s positive and negative similarity histograms to approximate the teacher’s better-separated histograms, and the order loss explicitly maximizes the expectation margin $\omega_{r_j} = r_j \hat{W}_j.$4 for both teacher and student, thereby reducing overlap between positive and negative distributions. The paper contrasts this with triplet, focal, OHEM, and classical KD-style methods such as SP and RKD, arguing that distribution-level alignment is less sensitive to noisy sampling.

6. Optimization details, empirical findings, and limitations across the two usages

For ArcFace+DDL, the training pipeline uses two data pools—easy $\omega_{r_j} = r_j \hat{W}_j.$5 and hard $\omega_{r_j} = r_j \hat{W}_j.$6—with batch construction built from positive pairs and singletons for hard-negative mining (Huang et al., 2020). On SCface the paper uses $\omega_{r_j} = r_j \hat{W}_j.$7 and per-GPU batch size $\omega_{r_j} = r_j \hat{W}_j.$8, reflecting one teacher plus two student distributions for $\omega_{r_j} = r_j \hat{W}_j.$9 and $i$00; on other datasets $i$01 and per-GPU batch size $i$02. The backbone choices are ResNet-50 and ResNet-100 as in ArcFace; optimization uses SGD, momentum $i$03, weight decay $i$04, and learning rate $i$05 divided by $i$06 at half the iterations. Hardware is $i$07 NVIDIA Tesla P40 GPUs with a TensorFlow implementation. Additional training cost comes from pair-similarity computation, soft-histogram estimation, and KL/order losses, with no additional inference-time cost.

Empirically, ArcFace+DDL improves especially on hard subsets. On SCface Rank-1 identification, ArcFace (CASIA+R50)-FT yields average $i$08 with $i$09, $i$10, $i$11, while DDL reaches average $i$12 with $i$13, $i$14, $i$15. For ArcFace (MS1M+R100)-FT the average is $i$16 with $i$17, $i$18, $i$19, and DDL reaches $i$20 with $i$21, $i$22, $i$23. On pose benchmarks with VGGFace2 pre-training, ArcFace (VGG+R100) obtains CFP-FP $i$24 and CPLFW $i$25, whereas DDL yields CFP-FP $i$26 and CPLFW $i$27. On IJB-B and IJB-C, the paper reports higher TAR at $i$28 and $i$29, as well as higher FPIR$i$30 and Rank-1 in $i$31 mixed-media settings (Huang et al., 2020).

The ablations are structurally informative. On SCface, KL+Order is best at average $i$32, compared with JS+Order $i$33 and EMD+Order $i$34. Hard-negative mining improves average performance from $i$35 with random negatives to $i$36. Modeling $i$37 and $i$38 as separate student distributions outperforms mixing them into one, with $i$39 versus $i$40. The paper also reports that on COX, DDL achieves comparable results to ArcFace finetuning with only $i$41 of training subjects.

Both literatures also state limitations. For HyperSpaceX DistArc, performance depends on sensible radius assignment, gains may be smaller in extremely high-dimensional embeddings, and excessive $i$42 can cause radial terms to overshadow angular margins (Chiranjeev et al., 2024). For ArcFace+DDL, benefits diminish if easy and hard distributions are not meaningfully different, the order-loss weight $i$43 is sensitive, histogram parameters $i$44 and $i$45 can destabilize gradients if chosen poorly, hard-negative mining depends on batch composition, and substantial label noise degrades the estimated distributions despite dropping positive pairs with negative similarity (Huang et al., 2020).

A final source of confusion is lexical rather than methodological. “Distributional adversarial loss” is a different concept entirely: it defines adversarial risk over families of perturbation distributions and studies PAC-learning guarantees, randomized smoothing, and derandomization (Ahmadi et al., 2024). It is unrelated to the HyperSpaceX DistArc loss except for the shared use of distributional language.

Taken together, the literature supports a bifurcated understanding of DistArc Loss. In its official 2024 usage, it is a radial–angular cross-entropy objective for multi-hyperspherical representation learning that combines $i$46, $i$47, and radial distance penalties (Chiranjeev et al., 2024). In its informal 2020 usage, it denotes ArcFace regularized by Distribution Distillation Loss, where easy-sample similarity distributions serve as teachers for hard-sample distributions through KL alignment and expectation-gap maximization (Huang et al., 2020). The two approaches address different bottlenecks—angular crowding in one case, hard-sample generalization in the other—even though both build on ArcFace-era embedding geometry.