Class-Center Similarity Explained

• Class-Center Similarity (CCS) is a framework where predictions and regularizations are driven by the similarity between feature embeddings and adaptive class centers.
• It is applied across varied tasks like semantic segmentation, contrastive learning, fine-grained visual classification, and deep hashing to improve intra-class compactness and inter-class separability.
• By replacing conventional logit-based reasoning with similarity measures, CCS methods enhance model performance through structured representation space and tailored loss functions.

Searching arXiv for the cited papers and closely related work on class-center similarity. Class-Center Similarity (CCS) denotes a family of formulations in which prediction, supervision, or regularization is expressed through the relation between a feature representation and one or more class centers. A class center may be realized as a classifier weight vector, a learnable prototype, a running-average feature prototype, or a batch-estimated binary prototype, depending on the task and optimization regime. The term is not fully standardized across the literature: in semantic segmentation, CCS appears explicitly as a Class Center Similarity layer that reframes pixel-wise prediction as pixel-to-center similarity (Wu et al., 2023); in contrastive-center learning, CCS can be defined through distances or contrastive ratios tied to class centers (Qi et al., 2017); in fine-grained visual classification, the same underlying idea is implemented through sample-center and center-center cosine similarity without being formalized as a standalone term (Yao et al., 2024); and in deep hashing, an analogous mechanism is presented as centers similarity learning rather than CCS (Zhang et al., 2021).

1. Conceptual scope and formalization

At a high level, CCS replaces or augments direct class-logit reasoning with similarity to class representatives in embedding space. The general structure is consistent across tasks: a feature $x$ is compared against class centers $\{c_k\}$, and the training objective encourages high similarity to the target center together with low similarity to non-target centers. What changes across instantiations is the definition of the center, the similarity metric, and the update rule.

Setting	Class center representation	Similarity mechanism
Semantic segmentation	Global classifier weights $w_j$ or scene-conditioned adaptive centers $c_k$	Inner product for pixel-to-center, cosine for center-to-center
Contrastive-center learning	Learnable center $c_k$ per class	Squared Euclidean distance, contrastive ratio
Fine-grained visual classification	Running-average class-center feature $F_j$	Cosine similarity for sample-center and center-center
Deep hashing	Learnable center $\mu_j$ and batch-wise binary prototype $u_j$	Inner product or cosine via $\theta_{ij}$ and sigmoid likelihood

In the segmentation formulation, the conventional $1\times 1$ convolution head already contains an implicit center structure: each weight vector of the segmentation head represents its corresponding semantic class in the whole dataset and can be regarded as the embedding of the class center. Pixel-wise classification then amounts to computing similarity between a pixel embedding and the class centers. This interpretation turns the classifier into a prototype matcher and motivates scene-conditioned center generation (Wu et al., 2023).

In contrastive-center learning, the center is a learnable parameter updated together with the network. The relevant quantity is not merely closeness to the target center, but a contrastive relation between the target-center distance and the aggregate distance to non-target centers. This yields CCS definitions such as

$\{c_k\}$0

and

$\{c_k\}$1

which encode intra-class compactness and inter-class separability in a single ratio (Qi et al., 2017).

Fine-grained visual classification adopts a different center semantics. There, a class center is a feature prototype $\{c_k\}$2 representing the whole class, maintained by a running average of sample embeddings rather than gradient descent. CCS is instantiated as cosine similarity between a sample and its class center, and between class centers themselves. The most similar non-target class is selected by an $\{c_k\}$3 over the center-center similarity matrix, which makes CCS a mechanism for identifying and penalizing the hardest confusions (Yao et al., 2024).

In deep supervised hashing, the center concept splits into two objects: learnable class centers $\{c_k\}$4 and batch-wise estimated binary prototypes $\{c_k\}$5. Here CCS is a logistic likelihood over center similarity scores

$\{c_k\}$6

or, in a continuous relaxation, via cosine. This couples classwise sample-to-center attraction with prototype-to-prototype concentration and repulsion (Zhang et al., 2021).

A common misconception is to treat CCS as a single loss or a single metric. The literature instead shows a broader design pattern: CCS may be an inference layer, an auxiliary loss, a prototype update rule, or a center-to-center regularizer, and it may use inner product, cosine similarity, or squared Euclidean distance depending on the geometry of the task.

2. Pixel-to-center similarity in semantic segmentation

The segmentation formulation begins from two stated difficulties: large intra-class feature variation in different scenes and small inter-class feature distinction in the same scene. The central observation is that standard semantic segmentation already computes a form of CCS. Let the backbone output a feature map $\{c_k\}$7, where the feature of pixel $\{c_k\}$8 is $\{c_k\}$9. If the conventional segmentation head has $w_j$0 class weights $w_j$1, with $w_j$2-th row $w_j$3, then the per-pixel, per-class score is

$w_j$4

Under the class-center view, each $w_j$5 is a global class center learned across the dataset, and the classifier is a pixel-to-center similarity calculator (Wu et al., 2023).

The Class Center Similarity layer replaces the final $w_j$6 convolution head with a three-stage pipeline. First, the Adaptive Class Center Module (ACCM) generates scene-conditioned adaptive centers $w_j$7. Second, the Similarity Calculation Module computes pixel-to-center and center-to-center similarities. Third, Class Distance losses increase inter-class separation and intra-class compactness at the scene level. ACCM aggregates scene features into coarse centers using a learned spatial weight map $w_j$8:

$w_j$9

where $c_k$0 yields $c_k$1 coarse centers by weighted spatial pooling per class channel, and $c_k$2 is implemented as a $c_k$3 convolution that refines them into adaptive centers. The mask $c_k$4 is predicted from $c_k$5 and supervised by a Dice loss.

Prediction uses inner-product pixel-to-center similarity:

$c_k$6

followed by

$c_k$7

For center-to-center similarity, the formulation uses cosine similarity with an absolute value in the numerator:

$c_k$8

The absolute value is introduced to encourage linear independence and remove norm influence.

The loss is built from a relative similarity

$c_k$9

and a distance

$c_k$0

The scene-level intra-class loss is

$c_k$1

With inner-product similarity and softmax normalization, this recovers the standard cross-entropy form; cross-entropy is therefore a special case of the proposed intra-class distance loss. The scene-level inter-class loss penalizes high similarity between different adaptive centers:

$c_k$2

The complete objective is

$c_k$3

with

$c_k$4

This construction makes CCS both an inference mechanism and a structured regularizer. ACCM adapts class centers to scene content, the intra term contracts per-class pixel clouds toward scene-specific centers, and the inter term pushes centers apart inside the current image. The geometric interpretation given in the source is that pixels and class centers lie in an embedding space, adaptive centers align to local scene distributions, and clearer class-separation hyperplanes emerge as intra-class contraction and inter-class expansion proceed jointly (Wu et al., 2023).

3. Contrastive-center formulations in discriminative representation learning

In contrastive-center learning, CCS is grounded in a class-center objective rather than a prediction head. Let $c_k$5 be the deep feature of sample $c_k$6, $c_k$7 its label, and $c_k$8 the learnable center for class $c_k$9. Using squared Euclidean distance, define

$F_j$0

with $F_j$1 by default to prevent division by zero. The contrastive-center loss is

$F_j$2

Training uses the joint supervision

$F_j$3

where $F_j$4 is the softmax cross-entropy (Qi et al., 2017).

Within this formulation, CCS can be defined in several ways. A pure target-center similarity is

$F_j$5

A contrastive CCS that explicitly includes non-target centers is

$F_j$6

and a loss-aligned CCS is

$F_j$7

The source identifies $F_j$8 as the CCS most directly tied to the optimization objective and $F_j$9 as a larger-is-better separation-focused metric.

The gradient structure makes the intended geometry explicit. For the per-sample contribution $\mu_j$0, the gradient with respect to the feature is

$\mu_j$1

When $\mu_j$2, center $\mu_j$3 is attracted toward $\mu_j$4; when $\mu_j$5, $\mu_j$6 is repelled from $\mu_j$7 proportionally to $\mu_j$8. The center update is

$\mu_j$9

with a smaller dedicated learning rate $u_j$0 recommended for stability.

The relation to prior losses is sharply defined. Softmax cross-entropy optimizes classification boundaries but does not explicitly penalize distances to centers or enforce center separation. Center loss

$u_j$1

improves intra-class compactness but ignores inter-class separability. Contrastive and triplet losses operate on pairs and triplets and require selection or mining. By contrast, contrastive-center loss uses class centers rather than pairs or triplets and explicitly balances intra-class compactness against inter-class separation with per-batch $u_j$2 cost (Qi et al., 2017).

This framing suggests a broad interpretation of CCS: it need not be a similarity function in the narrow sense of a bounded score; it can also be a contrastive ratio or negative loss term that is monotone with desirable center geometry.

4. Fine-grained visual classification and exploration of class centers

Fine-grained visual classification uses class centers to address two stated problems: evident intra-class variances and subtle inter-class differences, together with overfitting from fewer training samples. The formulation defines a class center as a feature prototype $u_j$3 representing class $u_j$4. Unlike center-loss-based methods, these centers are not learnable parameters updated by gradient descent. Instead, each center is maintained by a running average with a per-class counter:

$u_j$5

Centers are randomly initialized before training and then deterministically updated per incoming sample (Yao et al., 2024).

Cosine similarity is used both for sample-center and center-center relations:

$u_j$6

With $u_j$7 classes, the center-center similarity matrix is $u_j$8 with entries

$u_j$9

For sample $\theta_{ij}$0 of class $\theta_{ij}$1, the most similar non-target class is selected as

$\theta_{ij}$2

and $\theta_{ij}$3 serves both as a hardest-negative selector and as an adaptive weight.

The feature-side component is the multiple class-center constraint:

$\theta_{ij}$4

Equivalently,

$\theta_{ij}$5

Minimizing this pulls each sample toward its target center while pushing it away from the most similar non-target center. The non-target term is stronger when the target class center is itself highly similar to another center, so repulsion is focused where inter-class confusion is greatest.

The label-side component is class-center label generation. For each class, the method maintains a running average of the final linear-layer logits:

$\theta_{ij}$6

Then

$\theta_{ij}$7

and the regularization term is

$\theta_{ij}$8

The total objective augments standard cross-entropy:

$\theta_{ij}$9

Two aspects distinguish this CCS formulation from earlier center-based losses. First, the centers are stabilized by averaging rather than by gradient updates, which the source associates with smooth, stable center evolution compared to large oscillations seen with center loss and its variants. Second, CCS operates simultaneously at the feature level and the label level: the multiple class-center constraint reduces intra-class variance and enlarges inter-class differences, while class-center label generation produces soft labels that reflect class-center distributions rather than uniform label smoothing (Yao et al., 2024).

The source explicitly notes that the paper does not formally name “Class-Center Similarity” as a standalone construct, but also states that a precise CCS formulation consistent with the method is intrinsic to the loss. In that sense, CCS here is less a named module than a unifying geometric principle.

5. Centers similarity learning for deep supervised hashing

In deep supervised hashing, CCS appears as centers similarity learning. The setting includes a feature embedding $1\times 1$0, a binary hash code $1\times 1$1, learnable class centers $1\times 1$2, and batch-wise estimated binary prototypes $1\times 1$3. The classwise objective uses a Gaussian-softmax form:

$1\times 1$4

This term pulls samples toward their class centers in Euclidean space (Zhang et al., 2021).

CCS proper is defined through a similarity likelihood between batch-wise binary prototypes and learnable centers. Let

$1\times 1$5

with an alternative interpretation through cosine,

$1\times 1$6

Define $1\times 1$7 to indicate whether $1\times 1$8 and $1\times 1$9 correspond to the same class, and let $\{c_k\}$00. The likelihood is

$\{c_k\}$01

The resulting CCS loss is the negative log-likelihood

$\{c_k\}$02

A quantization penalty enforces proximity to the discrete codes:

$\{c_k\}$03

The full objective is

$\{c_k\}$04

The method uses a two-step strategy. In the first step, a batch-wise binary center is estimated for each class appearing in the mini-batch by solving

$\{c_k\}$05

which is equivalent to bitwise voting with the closed-form solution

$\{c_k\}$06

Running sums are maintained across iterations. In the second step, the learnable centers are concentrated toward their own batch-estimated prototypes and repelled from prototypes of other classes through $\{c_k\}$07.

The gradient of the CCS term with respect to a learnable center is

$\{c_k\}$08

This logistic form is described as providing stable gradients: for mismatched pairs, the gradient magnitude is controlled by $\{c_k\}$09, which helps prevent degenerate collapse. The overall complexity is $\{c_k\}$10 per batch, compared with $\{c_k\}$11 or $\{c_k\}$12 behavior in pairwise or triplet hashing (Zhang et al., 2021).

CCS in this setting is therefore not a direct sample-to-class similarity for prediction. It is a prototype-to-prototype regularizer that reshapes the class-center geometry underlying the classwise hash loss.

6. Empirical behavior, limitations, and open directions

The empirical record attached to CCS is task-specific but internally consistent. In semantic segmentation, the CCS layer improves all tested heads on ADE20K with a ResNet-50 backbone: baseline FCN/PSP/DeepLabV3+ obtain $\{c_k\}$13 mIoU, while adding CCS yields $\{c_k\}$14 mIoU. On ADE20K validation with ResNet-101, DeepLabV3+ reaches $\{c_k\}$15 mIoU and CCSNet reaches $\{c_k\}$16 mIoU, a gain of $\{c_k\}$17; on Pascal Context with 59 classes, OCRNet reaches $\{c_k\}$18 mIoU and CCSNet reaches $\{c_k\}$19 mIoU. Ablations further show that Dice loss is the best supervision for the mask $\{c_k\}$20, that the best reported configuration on ResNet-50 FCN is $\{c_k\}$21, and that a ground-truth mask upper bound produces $\{c_k\}$22 mIoU and $\{c_k\}$23 Acc, indicating substantial headroom when adaptive centers are perfectly localized. Qualitative evidence includes t-SNE visualizations showing compressed per-class clusters and enlarged margins between confusing classes such as “tree” and “plant,” as well as pixel-to-center distance histograms showing smaller intra-class distances and larger inter-class distances with adaptive centers than with global mean centers (Wu et al., 2023).

In contrastive-center learning, the reported results are likewise aligned with the intended geometry. On MNIST with $\{c_k\}$24, contrastive-center loss improves accuracy to $\{c_k\}$25 versus softmax at $\{c_k\}$26 and center loss at $\{c_k\}$27; the visualization shows class centers are much more separated, with average $\{c_k\}$28 distance to the “center of centers” of approximately $\{c_k\}$29, about $\{c_k\}$30–$\{c_k\}$31 larger than with center loss. On CIFAR10, ResNet-20 with contrastive-center loss achieves $\{c_k\}$32, above softmax at $\{c_k\}$33 and center loss at $\{c_k\}$34. On LFW, FRN trained with contrastive-center loss achieves $\{c_k\}$35, compared with $\{c_k\}$36 for softmax-only and $\{c_k\}$37 for a retrained center-loss model (Qi et al., 2017).

In fine-grained visual classification, the exploration-of-class-center loss yields consistent improvements across datasets and backbones. With a ResNet50 backbone, the baseline versus ECC results are AIR $\{c_k\}$38, CUB $\{c_k\}$39, CAR $\{c_k\}$40, and NAB $\{c_k\}$41. The method also augments existing fine-grained pipelines: for example, CAL with ResNet101 improves from AIR $\{c_k\}$42 to $\{c_k\}$43 and CUB $\{c_k\}$44 to $\{c_k\}$45, while Swin Transformer improves from CUB $\{c_k\}$46 to $\{c_k\}$47. Ablations show that both the multiple class-center constraint and class-center label generation contribute, with the combined ECC objective performing best. The source also identifies failure modes: early unreliability of centers and distributions, the need to keep $\{c_k\}$48 smaller than $\{c_k\}$49 early in training, and the $\{c_k\}$50 cost of the center-center similarity matrix, which can be mitigated by periodic recomputation (Yao et al., 2024).

In deep hashing, IDCWH consistently surpasses DCWH across multiple datasets. On CIFAR-10 under the “mini” protocol, IDCWH achieves mAP values of $\{c_k\}$51 at $\{c_k\}$52 bits; under the “full” protocol, it reaches $\{c_k\}$53; on CIFAR-100, it achieves $\{c_k\}$54; and on MS-COCO, $\{c_k\}$55 at $\{c_k\}$56 bits. Removing CCS degrades mAP across all datasets and code lengths, with gains more pronounced at shorter code lengths. The source highlights practical limitations involving noisy prototype estimates under class imbalance, the growth of $\{c_k\}$57 softmax cost when the number of classes becomes large, and sensitivity to noisy labels (Zhang et al., 2021).

Taken together, these results support a common interpretation: CCS is a center-aware structuring principle for representation space. It may act through adaptive centers, contrastive ratios, cosine hardest-negative selection, or prototype-to-prototype logistic regularization, but in each case the central objective is the same: reduce intra-class variation while enlarging inter-class differences. A plausible implication is that CCS is best understood not as a single algorithm but as a reusable geometric template spanning dense prediction, classification, metric learning, and retrieval. Open directions explicitly proposed in the segmentation literature include transformer or hybrid backbones, dynamic center update strategies such as EMA of centers across scenes or memory banks, contrastive pretraining on pixel-center pairs and center-center negatives, and adaptive temperature scaling in the softmax for relative similarity (Wu et al., 2023).