---
title: 'Cluster Purge Loss: Concepts & Applications'
url: https://www.emergentmind.com/topics/cluster-purge-loss
type: topic
---

# Cluster Purge Loss: Concepts & Applications

Searching arXiv for papers on “Cluster Purge Loss” and closely related terminology.
to=arxiv_search.search ／久久json
{"query":"all:\"Cluster Purge Loss\" OR all:\"Cluster Purging\" OR all:\"Progressive Cluster Purification\"","max_results":10,"sort_by":"submittedDate","sort_order":"descending"} 天天中彩票腾讯ജ്assistant to=arxiv_search.search
Searching arXiv for the specific papers and terms relevant to the topic.
to=arxiv_search.search  йеңиjson
{"query":"ti:\"Cluster Purge Loss\" OR ti:\"Cluster Purging\" OR ti:\"Progressive Cluster Purification\"","max_results":10,"sort_by":"relevance","sort_order":"descending"}
to=arxiv_search.search  天天彩票与你同行json
{"query":"2007.02577 OR 2302.11234 OR 2507.20078 OR 1905.11787","max_results":10,"sort_by":"relevance","sort_order":"descending"}
“Cluster Purge Loss” denotes different but structurally related ideas across the arXiv literature. In machine learning, it appears most explicitly as a deep metric learning objective for structuring transformer embeddings in equivalent mutant detection, and more broadly as an interpretive label for objectives that purge unreliable cluster members, isolate outliers, or induce redundancy so that clustered elements can be removed safely. In astronomy, the same phrase is used in the ordinary sense of stellar mass loss from bound clusters. This suggests that the term is not a single canonical loss, but a cross-domain pattern in which selective removal from a clustered structure is made central to either optimization or physical interpretation [2507.20078].

## 1. Terminological scope and major usages

The literature attached to the phrase spans several distinct problem settings. In one line of work, “Cluster Purge Loss” is the exact name of a loss function introduced for equivalent mutant detection with UniXCoder. In another, no loss is literally named that way, but cluster purification or purging changes which samples contribute to a cluster-wise objective. Elsewhere, the phrase maps to rate–distortion outlier detection, filter pruning, or stellar-cluster mass loss.

| Domain | Paper | Usage |
|---|---|---|
| Equivalent mutant detection | “Cluster Purge Loss: Structuring Transformer Embeddings for Equivalent Mutants Detection” [2507.20078] | Explicit loss name |
| Unsupervised feature learning | “Progressive Cluster Purification for Unsupervised Feature Learning” [2007.02577] | Purge implemented through cluster purification and gated supervision |
| Outlier detection | “Cluster Purging: Efficient Outlier Detection based on Rate-Distortion Theory” [2302.11234] | Purging criterion derived from rate–distortion representivity |
| Convnet pruning | “Online Filter Clustering and Pruning for Efficient Convnets” [1905.11787] | Cluster loss induces redundancy so clustered filters can be purged |
| Stellar clusters | “Globular Cluster Mass Loss in the Context of Multiple Populations” [1507.05634]; “Evidences of tidal distortion and mass loss from the old open cluster NGC 6791” [1511.00073] | “Purge loss” as astrophysical mass loss |

A recurrent misconception is that “Cluster Purge Loss” always denotes a single, standardized objective. The PCP paper is explicit that it does **not** introduce a loss literally called “Cluster Purge Loss”; the exact terminology is Cluster Purification, composed of unreliable sample filtering and unstable sample filtering, with the training objective named the PCP loss [2007.02577]. The outlier-detection paper similarly does not name a quantity “Cluster Purge Loss,” although its representivity criterion admits a direct Lagrangian interpretation that functions as one [2302.11234].

## 2. Explicit Cluster Purge Loss in equivalent mutant detection

The most literal usage appears in equivalent mutant detection, where the task is to determine, for each mutant of an original program, whether the mutant is semantically equivalent to its origin. Mutants are grouped by their origin program; each origin defines a class, and the problem is therefore intra-class rather than primarily inter-class. The paper argues that standard fine-tuning with a classification head and cross-entropy emphasizes decision boundaries but does not thoroughly structure the embedding space to reflect nuanced intra-class semantic relationships. Cluster Purge Loss addresses this by organizing embeddings around the origin embedding, treated as the class center, and by enforcing dynamically adjusted borders derived from empirical distance statistics [2507.20078].

For each class \(c\), the method maintains two adaptive borders, called verges. The positive verge \(v_c^+\) is an EMA of distances from the origin to equivalent mutants, and the negative verge \(v_c^-\) is an EMA of distances from the origin to non-equivalent mutants. Distances use normalized cosine distance:
$$
\operatorname{dist}(a,b)=1-\frac{\operatorname{cossim}(a,b)+1}{2}.
$$
With smoothing factor \(s=2/(\gamma+1)\), the EMA recursion is
$$
\operatorname{EMA}_{n+1}=\operatorname{EMA}_n\cdot(1-s)+x\cdot s,
$$
and over a block of \(h\) new values,
$$
\operatorname{EMA}_{n+h}=\operatorname{EMA}_n\cdot(1-s)^h+s\cdot\sum_{j=1}^{h}x_j\cdot(1-s)^{h-j}.
$$

The loss itself is hinge-like and asymmetric:
$$
L_{\mathrm{CPL}}=\frac{1}{m}\sum_{i=1}^{m}\Big([\,\operatorname{dist}(o_{k_i},s_i)-v_{k_i}^{-}+\zeta\,]_+^{\alpha}\cdot l_i+[\,v_{k_i}^{+}-\operatorname{dist}(o_{k_i},s_i)+\zeta\,]_+^{\beta}\cdot(1-l_i)\Big),
$$
where \(l_i=1\) denotes an equivalent mutant and \(l_i=0\) a non-equivalent mutant. The total objective is
$$
L=L_{\mathrm{CE}}+\lambda\cdot L_{\mathrm{CPL}}.
$$
The interpretation is asymmetric. Equivalent mutants are penalized when they drift beyond territory summarized by the negative verge; non-equivalent mutants are penalized when they remain inside, or too close to, territory summarized by the positive verge. The paper uses \(\alpha=2\) and \(\beta=1/2\), so the two terms grow differently. Borders are stored as per-class buffers updated by EMA rather than by backpropagation.

The implementation uses UniXCoder in encoder mode with Graph-Guided Masked Attention, L2-normalized CLS embeddings, a RoBERTa-style classifier head, batch size \(4\), and \(30\) training epochs. The reported grid search explores \(\zeta\in[-0.06,0.01]\) with step \(0.01\) and \(\lambda\in[1.00,1.30]\) with step \(0.05\), while fixing \(\gamma=12\). On Java equivalent mutant detection, the best CPL configuration reaches \(P=95.31\%\), \(R=85.41\%\), and \(F1=89.46\%\) at \(\lambda=1.15\) and \(\zeta=-0.05\), compared with \(F1=87.22\%\) for the \(30\)-epoch CE baseline and \(F1=88.18\%\) for CE plus adapted contrastive loss. On the C dataset, the best CPL configuration reaches \(P=97.30\%\), \(R=95.52\%\), and \(F1=96.38\%\), again outperforming both baselines. The paper also reports that the median \(F1\) over the explored grid is \(87.52\%\) for CPL versus \(86.65\%\) for contrastive loss, with \(p<0.0001\), and that CPL increases the ratio of non-equivalent to equivalent mean distance from \(0.95\) to \(2.11\), indicating that the purge effect primarily pushes non-equivalents outward while leaving equivalents comparatively stable [2507.20078].

## 3. Purge as gated supervision in Progressive Cluster Purification

In unsupervised feature learning, Progressive Cluster Purification provides a different realization of the purge idea. The method does not define a separate “Cluster Purge Loss,” but the purification stage explicitly excludes or reassigns class-inconsistent samples before the loss is formed. The resulting PCP loss is therefore a purge-gated objective in which only reliable cluster members contribute to cluster supervision, while purged samples are either treated as individual-instance classes or excluded entirely [2007.02577].

The pipeline uses standard CNN backbones, with ResNet-18 and AlexNet reported in experiments and embedding dimensionality \(D=128\). At epoch \(t\), features \(v_i(t)=f_{\theta_t}(x_i)\in\mathbb{R}^D\) are clustered by k-means into \(N_t\) clusters. The number of clusters follows a logarithmic linear schedule,
$$
\log(N_t)=\Bigl(1-\frac{t}{T}\Bigr)\log(N),
$$
with \(T=200\), then is clamped at a floor \(N_{t_0}\) by
$$
N_t\leftarrow \max(N_t,N_{t_0}),
$$
with \(N_{t_0}\approx1000\) by default. This progressively reduces cluster count while cluster sizes expand as representations improve.

Purification has two stages. The first, unreliable sample filtering \(\mathrm{CP}_r\), assumes that samples far from a centroid are more likely class-inconsistent. For cluster \(S_c(t)\), the centroid is
$$
\mu_c(t)=\frac{1}{|S_c(t)|}\sum_{i\in S_c(t)}v_i(t),
$$
and deviation is measured by Euclidean distance
$$
d_i^c(t)=\|v_i(t)-\mu_c(t)\|_2.
$$
Given filtering ratio \(\gamma\in[0,1]\), the method keeps the closest
$$
m_c(t)=\lceil(1-\gamma)|S_c(t)|\rceil
$$
samples as reliable and purges the remainder:
$$
\mathcal{S}_c^r(t)=\operatorname{TopK}_{i\in S_c(t)}(-d_i^c(t),m_c(t)),\qquad
\mathcal{N}_c^r(t)=S_c(t)\setminus \mathcal{S}_c^r(t).
$$
The second stage, unstable sample filtering \(\mathrm{CP}_s\), uses temporal voting relative to the sample closest to the centroid,
$$
i_c(t)=\arg\min_{i\in S_c(t)}d_i^c(t),
$$
with score
$$
V\!\bigl(v_i(t),v_{i_c}(t)\bigr)=\sum_{k=0}^{n}\alpha^k\cdot\boldsymbol{\delta}\!\left(C(v_i(t-k)),C(v_{i_c}(t-k))\right),
$$
where \(\boldsymbol{\delta}(x,y)=1\) if \(x=y\) and \(-1\) otherwise. Reliable samples with \(V<\theta_S\) are discarded; unreliable samples with \(V\ge\theta_N\) are pulled back. In implementation, \(\mathrm{CP}_s\) starts after \(100\) epochs, uses \(n=15\), \(\theta_S=0\), and \(\theta_N=3.0\).

The PCP loss combines an instance-wise term over purged samples and a cluster-wise term over purified cluster members:
$$
L_{\mathrm{instance}}^t=-\sum_{i\in\mathcal{N}^s(t)}\log P(i\mid v_i(t)),
$$
$$
L_{\mathrm{cluster}}^t=-\sum_{c=1}^{N_t}\sum_{i,j\in\mathcal{S}_c^s(t)}\log P(i\mid v_j(t)),
$$
$$
L_{\mathrm{pcp}}^t=L_{\mathrm{instance}}^t+L_{\mathrm{cluster}}^t.
$$
With a non-parametric softmax memory bank,
$$
P(i\mid v)=\frac{\exp(v_i^\top v/\tau)}{\sum_{j=1}^{n}\exp(v_j^\top v/\tau)},
$$
the purge can be written explicitly as indicator gating:
$$
L_{\mathrm{pcp}}^t=
-\sum_{c=1}^{N_t}\sum_{i,j}\mathbf{1}[i,j\in\mathcal{S}_c^s(t)]\log P(i\mid v_j(t))
-\sum_{i}\mathbf{1}[i\in\mathcal{N}^s(t)]\log P(i\mid v_i(t)).
$$
This formulation makes the purge effect exact: purged samples no longer act as positives in the cluster term. If a sample is excluded from both \(\mathcal{S}^s\) and \(\mathcal{N}^s\), it contributes no gradient in that epoch. Empirically, on CIFAR10 with k-NN evaluation, the ablation reports \(73.6\%\) for the DC baseline, \(76.9\%\) with progressive clustering, \(78.9\%\) with \(\mathrm{CP}_r\), and \(81.6\%\) with \(\mathrm{CP}_s\) added; on ResNet-18, PCP reaches \(84.7\%\) in one round and \(87.3\%\) in five rounds [2007.02577].

## 4. Rate–distortion purging and the implicit purge loss

In rate–distortion outlier detection, Cluster Purging treats clustering as lossy compression and asks when a datum is better represented by a unique symbol than by its assigned cluster representative. The formalism begins with separable distortion,
$$
d(x,r)=\sum_{j=1}^{n}d(x_j,r_{c_j}),
$$
and an empirical rate–distortion function for deterministic clusterings,
$$
R(D,x,C):=\min_{\theta\in\Theta}h(c)\ \text{subject to}\ d(x,r)\le D,
$$
with empirical rate
$$
h(c)=-\sum_{g=1}^{\nu}\frac{f_g}{n}\log\frac{f_g}{n}.
$$
Because exact computation is infeasible for many clustering methods, the paper estimates the attainable trade-off through a lower convex hull over observed \((D_i,H_i)\) points, with piecewise-linear segments
$$
\mathcal{L}(D)=\kappa_i D+\delta_i.
$$
Representivity is then defined as
$$
\rho(x,c,r,C):=\frac{R(d(x,r),x,C)}{h(c)},
$$
and estimated for modified clusterings via the hull [2302.11234].

Purging a single point \(x_j\) assigns it to a singleton cluster with its own representative \(r^*\), satisfying \(d(x_j,r^*)=0\). The entropy and distortion changes are
$$
\Delta_e=h(c')-h(c)=\frac{1}{n}\Big[f_{c_j}\log f_{c_j}-(f_{c_j}-1)\log(f_{c_j}-1)\Big],
$$
$$
\Delta_d=d(x,r')-d(x,r)=-d(x_j,r_{c_j}).
$$
The decision rule states that \(x_j\) is a rate–distortion outlier if purging improves representivity across all clusterings on the hull, equivalently if
$$
d(x_j,r_{c_i,j})\ge \frac{h(c'_{(i,j)})-h(c_i)}{-\kappa_i}.
$$
Although the paper does not introduce a named “Cluster Purge Loss,” it gives a direct Lagrangian interpretation:
$$
L(c,r;\lambda)=h(c)+\lambda d(x,r),
$$
with \(\lambda=-\kappa_i>0\), and the per-point purge change
$$
\Delta L_{i,j}:=\Delta_e+\lambda\Delta_d=\Delta_e-\lambda d(x_j,r_{c_i,j}).
$$
Purging is beneficial exactly when \(\Delta L_{i,j}\le 0\). An aggregated form is
$$
L_{\mathrm{purge}}(j):=\max_i\Delta L_{i,j},
$$
and the point is marked as an outlier if \(L_{\mathrm{purge}}(j)\le 0\). This is the closest formal objective in the paper to a cluster purge loss.

The paper presents two algorithms. The parameter-free variant computes multiple clusterings, builds the rate–distortion hull, precomputes \(\Delta_e\) per cluster, and purges points whose purge margins are non-negative for all hull clusterings. The parametric variant uses a single clustering and a chosen \(\kappa\). Once clusterings are available, the purging step is \(O(n)\), with space \(O(tn)\) to store \(t\) clusterings. On \(13\) benchmark datasets, the paper reports average \(F1\) scores of approximately \(0.62\) for \(\mathrm{CPP}(\mathrm{HAC},\mathrm{DBSCAN})\), \(0.58\) for \(\mathrm{CPP}(\mathrm{k\text{-}means})\), and \(0.64\) for \(\mathrm{CBLOF}(\mathrm{HAC})\); it also reports that among detectors built on interchangeable clusterings, CPP did not perform worse than the respective vanilla clustering on \(100\%\) of datasets [2302.11234].

## 5. Redundancy-inducing cluster loss for filter purging

In convolutional network pruning, cluster purge is realized by inducing within-cluster redundancy so that one element per cluster can be retained while the others are removed safely. The paper “Online Filter Clustering and Pruning for Efficient Convnets” introduces an extra cluster loss term during training. Filters are partitioned into predefined clusters before training and the partition remains fixed; there is no online re-clustering. The purpose of the loss is to force filters in each cluster to become similar, after which one filter per cluster is kept, the others are pruned, and the pruned network is fine-tuned [1905.11787].

The total objective is written as
$$
f(K)=E(K)+\delta R(K)+\lambda\sum_{i=1}^{L}\sum_{t=1}^{T}\|k_i^t-c_i^t\|_2^2,
$$
with
$$
c_i^t=\frac{1}{|S_i^t|}\sum_{k_i^t\in S_i^t}k_i^t.
$$
An equivalent clarified form is
$$
L_{\mathrm{cluster}}=\sum_{i=1}^{L}\sum_{t=1}^{T_i}\sum_{w\in S_i^t}\|w-\mu_i^t\|_2^2,\qquad
\mu_i^t=\frac{1}{|S_i^t|}\sum_{w\in S_i^t}w,
$$
so that
$$
L_{\mathrm{total}}=L_{\mathrm{task}}+\delta R(K)+\lambda L_{\mathrm{cluster}}.
$$
For any filter \(w\in S_i^t\), the gradient is
$$
\frac{\partial L_{\mathrm{cluster}}}{\partial w}=2(w-\mu_i^t),
$$
which directly pulls the filter toward the cluster mean.

Cluster structure is fixed by the target pruning ratio \(p_i\le 0.5\) in each layer with \(N_i\) filters. The method creates \(N_i(1-p_i)\) clusters: \(N_ip_i\) clusters of size \(2\) and \(N_i(1-2p_i)\) singleton clusters. This exactly yields \(N_ip_i\) filters to prune. After training, one filter is kept from each size-\(2\) cluster. If two upstream filters have become equal, then their corresponding feature maps are equal, and for any downstream filter \(j\),
$$
y_{i+1}^j=h\big((k_i^{j1}+k_i^{j2})\otimes y_i^1+0\otimes y_i^2+\cdots+k_i^{jM}\otimes y_i^M\big).
$$
This establishes the merge-and-prune reparameterization: one channel is removed and the downstream weights are summed into the survivor. The per-layer speedup and compression are both
$$
r_{s_i}=r_{c_i}=(1-p_{i-1})(1-p_i).
$$

Training uses SGD with momentum \(0.9\), weight decay \(5\times10^{-4}\), batch size \(128\), and \(300\) epochs. The paper studies \(\lambda\) and uses \(\lambda=0.05\) in the main experiments. Reported unpruned accuracies are \(93.55\%\) and \(73.23\%\) for VGG-16 on CIFAR-10 and CIFAR-100, \(93.56\%\) and \(69.82\%\) for ResNet-34, and \(95.01\%\) and \(75.99\%\) for WRN-16-4. The paper highlights approximately \(2\times\) speedup for VGG-16 with no accuracy loss and approximately \(3.4\times\) speedup on CIFAR-10 for WRN-16-4 with approximately \(1\%\) accuracy drop [1905.11787]. In this setting, the “purge” is not sample rejection but structured removal enabled by a cluster loss that makes the removal functionally safe.

## 6. Astronomical usages: globular-cluster mass loss and open-cluster stripping

In astronomy, “cluster purge loss” refers to cumulative stellar mass loss from a bound stellar system through internal relaxation-driven evaporation and external tidal processes. Two distinct contexts appear in the cited literature. One concerns the viability of heavy early mass loss in globular-cluster multiple-population models; the other provides a direct observational case of ongoing mass loss in the open cluster NGC 6791 [1507.05634].

For globular clusters, the relevant quantity is the present enriched fraction among long-lived stars,
$$
f_{2,p}\equiv \frac{M_{2,p}}{M_{1,p}+M_{2,p}},
$$
where \(M_{1,i}\) and \(M_{2,i}\) denote initial masses in long-lived first-generation and second-generation stars, \(y\equiv M_{2,i}/M_{1,i}\), and \(l_1,l_2\) are fractional losses of first- and second-generation stars. Under the paper’s most favorable assumption for purge models, \(l_2\approx 0\), so
$$
f_{2,p}=\frac{y}{(1-l_1)+y},
$$
which yields
$$
l_1=1-y(f_{2,p}^{-1}-1).
$$
Across \(33\) Milky Way globular clusters, the compiled enriched fraction is \(f_{2,p}=0.68\pm0.07\). Adopting an initial enriched fraction of approximately \(0.05\), consistent with AGB/FRMS-style mass budgets, implies \(y\approx0.0526\), and reaching the observed enriched fraction requires roughly \(95\%-98\%\) loss of first-generation stars. For example, with \(f_{2,p}=0.68\), the paper gives \(l_1=1-0.470588\,y\), so \(y=0.05\) implies \(l_1\approx0.976\). The central result is that no significant correlations are found between the enriched fraction and present globular-cluster mass, Galactocentric radius, or metallicity, even though tidal stripping, gas expulsion, and birth-environment scenarios predict such correlations. The paper therefore concludes that globular clusters were likely not dramatically more massive at birth and that the present enriched-to-primordial fraction probably reflects the initial value rather than a global early purge [1507.05634].

For NGC 6791, the evidence points in the opposite direction: ongoing purge loss is directly observed. The cluster is described as unusually old, approximately \(8\) Gyr, metal-rich, and currently of mass approximately \(5000\,M_\odot\). Using CFHT/MegaCam imaging over a \(2^\circ\times2^\circ\) field, the authors find a clear elongation and irregular stellar distribution beginning at approximately \(300''\) from the center, and two tidal tails extending in opposite directions beyond the tidal radius. The tails and elongation are aligned with both the absolute proper motion direction and the Galactic-center direction. The star-count density profile follows a single-mass King model in the inner region but departs from it for \(r\gtrsim600''\), where the outer profile becomes a power law with slope \(\alpha\approx-1.7\). The paper interprets these features as evidence of ongoing mass loss driven by gravitational shocking and the Galactic tidal field [1511.00073].

The mass-loss estimate uses the Lamers et al. formalism,
$$
\frac{dM}{dt}=-\frac{M}{t_{\mathrm{dis}}},\qquad t_{\mathrm{dis}}=t_0M^\gamma,
$$
which integrates to
$$
M(t)=\mu_{\mathrm{se}}(t)\Big[M_0^\gamma-\gamma t/t_0\Big]^{1/\gamma}.
$$
Inverting gives
$$
M_0=\Big[\big(M(t)/\mu_{\mathrm{se}}\big)^\gamma+\gamma t/t_0\Big]^{1/\gamma}.
$$
For NGC 6791, the inferred initial mass is
$$
M_{\mathrm{ini}}=(1.5-4.0)\times10^5\,M_\odot.
$$
In this astronomical usage, purge loss is not an optimization device but the physical removal of stars through stellar evolution, relaxation-driven escape, disc and bulge shocking, and steady tidal stripping. The contrast between the globular-cluster paper and the NGC 6791 case is instructive: one rejects a universal heavy-purge interpretation for multiple populations, while the other documents a specific cluster that is currently undergoing significant purge loss [1511.00073].

Source: https://www.emergentmind.com/topics/cluster-purge-loss