Weight Similarity (WS): Methods and Applications

• Weight Similarity is a collection of techniques defining similarity through weighted coefficients, learnable functions, direct weight comparisons, or structural constraints.
• It applies in diverse contexts such as sparse data analysis, graph embedding, and neural network regularization, using measures like the weighted Jaccard similarity and DOCS.
• Practical implementations range from enhancing CNN robustness with spatial losses to reducing model complexity via weight sharing in variational graph autoencoders and hardware optimizations.

Searching arXiv for the cited papers to ground the article in the literature and verify identifiers. {"query":"id:(Kim et al., 2019) OR id:(Mollgaard et al., 2016) OR id:(Salha-Galvan et al., 23 Feb 2025) OR id:(Pourchot et al., 2020) OR id:(Min et al., 28 Jan 2025) OR id:(Zhang et al., 13 Jan 2025) OR id:(Truong et al., 31 Jul 2025) OR id:(Raff et al., 2018) OR id:(Wu et al., 2018) OR id:(1304.1084) OR id:(Jr. et al., 10 Aug 2025) OR id:(Wang et al., 2022) OR id:(Qiao et al., 2019)","max_results":12} Weight Similarity (WS) is not a single standardized construct in the arXiv literature. Across recent work, the phrase and the abbreviation denote several non-equivalent ideas: weighted coefficients over observations or attributes, learnable similarity functions in representation learning, direct comparison of neural-network weight matrices, and architectural or hardware constraints that force weights to align, repeat, or share structure (Kim et al., 2019, Zhang et al., 13 Jan 2025, Min et al., 28 Jan 2025, Truong et al., 31 Jul 2025). The common thread is that “weight” is either part of the similarity definition itself or the object whose similarity is being measured.

1. Terminological scope

In the literature considered here, WS is best understood as a family of related but distinct technical usages rather than a single canonical metric. Some usages define similarity between data objects, some define similarity functions for learned embeddings, and some compare or constrain the parameters of neural networks directly.

Usage	Object compared	Representative form
Weighted coefficient over data	attributes, sparse vectors, or linked nodes	$\mathrm{WJS}(S,O)=\frac{\sum \min}{\sum \max}$, $$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$, $r=t^2/s^2$
Learned similarity function	embeddings or graph nodes	$$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$
Direct weight comparison	neural-network weight matrices	DOCS, chain-normalized weight representations
Constraint on weights	neighboring units, encoder branches, or bit-columns	local $L_2$ regularization, hard parameter sharing, duplicate-column reuse

The ambiguity is substantive. In graph representation learning, “weighted inner product similarity” refers to a learnable indefinite inner product over embedding coordinates (Kim et al., 2019). In sparse NLP data, “weighted similarity” is an additive score that combines numerical overlap, shared missingness, and mismatched missingness (Zhang et al., 13 Jan 2025). In LLMs, weight similarity is a direct comparison between trained matrices, formalized by the Distribution of Cosine Similarity (DOCS) (Min et al., 28 Jan 2025). In topographic CNNs, Weight Similarity is a local regularizer that pushes neighboring units to develop similar incoming weights (Truong et al., 31 Jul 2025).

A useful practical distinction is between weighted similarity, where weights appear in the similarity formula, and similarity of weights, where trained parameters themselves are the compared objects. Several papers move between these meanings, but they should not be conflated.

2. Weighted similarity over observations, attributes, and missingness

One long-standing lineage defines WS as a coefficient over data vectors or pairwise observations. In weighted-set similarity, the canonical quantity is the Weighted Jaccard Similarity, also called the min-max kernel,

$$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$

or, in vector form,

$$S(\mathbf{x},\mathbf{y})= \frac{\sum_i \min(x_i,y_i)}{\sum_i \max(x_i,y_i)}.$$

This is the target similarity estimated by Consistent Weighted Sampling, ICWS, and SCWS, and it reduces to ordinary Jaccard similarity when all weights are equal (Raff et al., 2018, Wu et al., 2018).

A second formulation is designed for highly sparse vectors with many missing values. There, WS is explicitly decomposed into three components,

$$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon},$$

where $\mathrm{sNum}$ measures agreement on jointly observed entries, $\mathrm{sNan}$ rewards jointly missing entries, and $$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$0 penalizes one-sided missingness. For more than one common observed entry, the numerical term is overlap-scaled cosine similarity; for exactly one, it becomes a min/max ratio; and for no overlap it is set to zero (Zhang et al., 13 Jan 2025). This formulation is notable because shared absence is treated as informative similarity rather than discarded nuisance.

Mixed-type data introduce a different weighting problem. For Gower’s dissimilarity, the weighted form is

$$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$1

with $$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$2 handling missing values and $$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$3 defined variable by variable for numeric, binary, and categorical data (D'Orazio, 2024). The paper argues that the nominally “unweighted” choice $$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$4 hides an unbalanced contribution of variables, because categorical mismatches frequently contribute a full $$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$5 whereas range-scaled quantitative dissimilarities rarely do. The proposed remedy is automatic weight selection that minimizes the dispersion of the correlations between each variable-specific dissimilarity and the resulting weighted Gower dissimilarity (D'Orazio, 2024).

A related context-dependent approach appears in binary attribute spaces. There the dissimilarity is a weighted Hamming form,

$$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$6

with

$$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$7

so that attribute weights depend on the empirical frequency $$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$8 of value $$\mathrm{WS}=\mathrm{sNum}+\mathrm{sNan}+\mathrm{sNon}$$9 in the current context (1304.1084). Attributes near balance $r=t^2/s^2$0 receive larger weight than attributes that are almost always $r=t^2/s^2$1 or $r=t^2/s^2$2.

In network analysis, weighted similarity can also mean weighted homophily. The multilayer-network paper defines a weighted intraclass correlation coefficient

$r=t^2/s^2$3

where each linked pair contributes proportionally to its edge weight $r=t^2/s^2$4, and a tuning exponent $r=t^2/s^2$5 controls whether weak or strong ties dominate the measure (Mollgaard et al., 2016). This is explicitly a node-attribute similarity measure, not a structural node-similarity score.

3. Learned similarity functions and weighted graph representations

A separate usage treats WS as a learnable similarity model. The clearest example is weighted inner product similarity (WIPS), defined for embeddings $r=t^2/s^2$6 by

$r=t^2/s^2$7

with learnable coordinate weights $r=t^2/s^2$8, including negative values (Kim et al., 2019). In Siamese form,

$r=t^2/s^2$9

This generalizes IPS, subsumes SIPS and IPDS through explicit reparameterizations, and inherits the universal approximation capability of general similarities from IPDS. The theoretical point is that allowing negative $$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$0 yields an indefinite inner product associated with a pseudo-Euclidean space, making it possible to represent PD, CPD, and indefinite similarities within one parameterization (Kim et al., 2019).

The same paper gives a matrix-decomposition interpretation. For a symmetric similarity matrix $$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$1 with spectral decomposition

$$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$2

truncation to the leading $$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$3 eigenpairs gives

$$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$4

and the resulting approximation can be read as a weighted inner product over $$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$5-dimensional embeddings. This connects WS to a relaxed learnable eigendecomposition without orthogonality constraints (Kim et al., 2019).

In graph embedding, a different WS-like operationalization measures whether original edge weights are preserved by embedding similarity. The relevant score is the Pearson correlation between original edge weights and cosine similarity between endpoint embeddings,

$$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$6

computed over connected node pairs (Jr. et al., 10 Aug 2025). Weight-aware random walks consistently outperform unweighted and strength-based walks on this criterion, achieving correlations above $$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$7 in network models, although performance in real-world networks is much more heterogeneous (Jr. et al., 10 Aug 2025).

Weighted graph similarity also appears in kernel form. For complete weighted graphs, one paper first computes a commute-time minimum spanning tree, then probes it with a discrete-time quantum walk initialized by a weighted Perron-Frobenius operator, and finally compares the induced probability vectors with either

$$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$8

or

$$\langle y,y' \rangle_\lambda=\sum_k \lambda_k y_k y'_k$$9

Both resulting similarities are presented as positive-definite graph kernels (Bai et al., 2019). Here, weight information is not compared edgewise; it is re-encoded through commute time, tree extraction, and quantum-walk dynamics.

4. Direct similarity of neural-network weights

A more recent strand uses WS to compare trained neural-network parameters directly. In LLMs, DOCS defines similarity between two weight matrices $L_2$0 and $L_2$1 by first computing the cosine-similarity matrix

$L_2$2

then taking the maximum absolute cosine for each column,

$L_2$3

fitting Gumbel distributions to the two maxima vectors, and defining

$L_2$4

where $L_2$5 and $L_2$6 are the fitted location parameters (Min et al., 28 Jan 2025). DOCS is claimed to satisfy permutation transformation invariance, symmetry, isotropic scaling invariance, reflexivity, and discriminativity on orthogonal matrices (Min et al., 28 Jan 2025).

Empirically, DOCS reveals depth-wise organization in open-source LLMs. Adjacent layers frequently exhibit high weight similarity and tend to form clusters; base and instruction-tuned variants remain highly similar, with all reported DOCS scores above $L_2$7; and some MoE experts appear as outliers with much lower similarity to the rest (Min et al., 28 Jan 2025). The paper also stresses that representation similarity and weight similarity are distinct, especially in residual architectures where similar activations can arise from different underlying weights (Min et al., 28 Jan 2025).

A different direct-parameter approach handles permutation ambiguity explicitly. The chain normalization rule defines

$L_2$8

so that hidden-unit permutations cancel along the chain (Wang et al., 2022). Using these normalized weight representations, together with a hypothesis-training-testing procedure, the paper reports that identical neural networks optimized with SGD converge to a similar local solution in a metric space across MLP, CNN, and RNN settings (Wang et al., 2022). The crucial claim is not raw parameter equality, but equivalence after normalization.

5. Weight similarity as a constraint in architectures and hardware

Some papers use WS not as a measurement but as an explicit constraint on model structure. In topographic CNNs, Weight Similarity is a local spatial loss applied to the $L_2$9 grid of 121 units in the first fully connected layer. For each grid cell, the Moore neighborhood is identified, the $$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$0 norm is computed between the incoming weight vector of that cell and those of each neighboring cell, and the resulting average pairwise distance enters the joint loss

$$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$1

with $$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$2 (Truong et al., 31 Jul 2025). Compared with Activation Similarity and standard CNNs, WS improved robustness to noise, produced higher activation variance, and yielded stronger functional localization, with similarly responding units positioned at closer distances (Truong et al., 31 Jul 2025).

A nearby but distinct usage is weight sharing. In variational graph autoencoders, WS denotes hard equality of hidden-layer parameters across the mean and variance encoders, for example

$$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$3

This reduces parameter count and encoder expressivity simultaneously, but across 10 VGAE variants and 16 datasets the shared-weight models remain within one standard deviation of their non-shared counterparts, leading the authors to recommend WS as a practical default (Salha-Galvan et al., 23 Feb 2025). In neural architecture search, by contrast, WS denotes super-net parameter sharing across candidate architectures. There the evidence is more skeptical: moderately good proxy/standalone correlations do not reliably translate into better search than random search, and search-space-induced bias is reported as the dominant issue (Pourchot et al., 2020).

A hardware-oriented usage appears in RRAM-based accelerators. There, a bit-level weight reordering strategy based on column similarity operates on binary two’s-complement weight matrices. Similarity between two bit-columns is measured by

$$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$4

and the method “treats bit sparsity as a special case of bit similarity,” keeping only one column from a pair of columns that have identical bit values after row reordering (Yang et al., 18 Nov 2025). The reported result is a $$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$5 average performance improvement and $$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$6–$$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$7 energy savings under different sparsity ratios, with slight overhead relative to the compared design (Yang et al., 18 Nov 2025).

6. Conceptual distinctions and recurring caveats

A first recurring issue is terminological drift. “WS” may denote Weight Similarity, weight sharing, or Weight Standardization, depending on the paper. In micro-batch vision training, for example, WS means Weight Standardization: convolutional filters are normalized to zero mean and unit variance,

$$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$8

and the method is about optimization smoothness rather than similarity measurement (Qiao et al., 2019). The abbreviation alone is therefore not semantically reliable.

A second issue concerns what counts as informative absence. In the sparse-data metric, jointly missing entries contribute positively through $$\mathrm{WJS}(S,O)= \frac{\sum_{z \in S \cup O} \min\bigl(w(S,z),w(O,z)\bigr)} {\sum_{z \in S \cup O} \max\bigl(w(S,z),w(O,z)\bigr)},$$9, and one-sided missingness contributes negatively through $$S(\mathbf{x},\mathbf{y})= \frac{\sum_i \min(x_i,y_i)}{\sum_i \max(x_i,y_i)}.$$0 (Zhang et al., 13 Jan 2025). In Gower’s framework, missing values are instead excluded by $$S(\mathbf{x},\mathbf{y})= \frac{\sum_i \min(x_i,y_i)}{\sum_i \max(x_i,y_i)}.$$1, while asymmetric binary variables explicitly set $$S(\mathbf{x},\mathbf{y})= \frac{\sum_i \min(x_i,y_i)}{\sum_i \max(x_i,y_i)}.$$2 for the $$S(\mathbf{x},\mathbf{y})= \frac{\sum_i \min(x_i,y_i)}{\sum_i \max(x_i,y_i)}.$$3 case so that joint absence does not count as similarity (D'Orazio, 2024). This suggests that WS formulations differ not merely in weighting scheme but in the semantics assigned to absence and missingness.

A third distinction is geometric. In weighted inner product similarity, negative coordinate weights are essential because ordinary inner products are tied to positive-definite geometry; WIPS uses negative $$S(\mathbf{x},\mathbf{y})= \frac{\sum_i \min(x_i,y_i)}{\sum_i \max(x_i,y_i)}.$$4 to realize an indefinite inner product and thereby approximate similarities beyond the PD and CPD classes (Kim et al., 2019). In parameter-space analysis, conversely, the main difficulty is not kernel definiteness but invariance: weight similarity must be distinguished from representation similarity, and raw parameters may be incomparable because of permutation or residual-connection effects (Min et al., 28 Jan 2025).

Taken together, these usages show that WS is best treated as a technical umbrella term. In some settings it is a coefficient over sparse or mixed-type observations; in others it is a learnable similarity family, a direct weight-matrix comparison, or a structural regularizer that enforces redundancy. The literature does not converge on one universal definition, but it does converge on a narrower principle: weighting, whether in data space, embedding space, parameter space, or hardware mapping, is used to expose structure that uniform or unweighted formulations would miss.