--- title: Symmetric Average Minimum Distance (SAMD) url: https://www.emergentmind.com/topics/symmetric-average-minimum-distance-samd type: topic --- # Symmetric Average Minimum Distance (SAMD) Searching arXiv for the cited papers to ground the article in the relevant literature. Symmetric Average Minimum Distance (SAMD) denotes two distinct quantities in recent arXiv literature. In lexical semantic change detection (LSCD), SAMD is introduced as a measure that quantifies semantic change via local correspondence between word usages across time periods [2602.15716]. In coding theory, \(\mathrm{SAMD}(n,M)\) denotes the minimum average Hamming distance of binary codes with length \(n\) and size \(M\) [1910.09416]. The acronym therefore has domain-specific meanings: a one-to-one matching score over contextualised embeddings in lexical semantics, and an extremal quantity over binary codes in Hamming space. ## 1. Terminological scope and domain-specific meanings In LSCD, the relevant objects are two sets of contextualised embeddings for a target word \(w\) from two time periods. The purpose of SAMD in that setting is to quantify semantic change via local correspondence between word usages across time periods, in contrast to metrics that operate on global averages or prototypes [2602.15716]. In coding theory, the relevant object is a binary code \(A\subset\{-1,1\}^n\) of size \(M\). There, SAMD is not a matching algorithm or an embedding-space score, but the extremal quantity \[ \mathrm{SAMD}(n,M):=\min_{\substack{A\subset\{\pm1\}^n\\|A|=M}} D(A), \] where \(D(A)\) is the average Hamming distance of the code [1910.09416]. This suggests a terminological overlap rather than a shared formalism. The two uses of SAMD are mathematically independent, despite the identical acronym. ## 2. SAMD in lexical semantic change detection Let \(A=\{a_1,\dots,a_m\}\subset\mathbb{R}^D\) and \(B=\{b_1,\dots,b_n\}\subset\mathbb{R}^D\) be two sets of contextualised embeddings for a target word \(w\) from time periods 1 and 2. Using cosine distance \(\delta(x,y)=1-\cos(x,y)\), Symmetric Average Minimum Distance enforces a one-to-one matching between \(A\) and \(B\) after sampling both sets to the same size \(N=\min(m,n)\) [2602.15716]. Denote the selected subsets by \(A'=\{a_{i_1},\dots,a_{i_N}\}\) and \(B'=\{b_{j_1},\dots,b_{j_N}\}\). Let \(M=\{(p_1,q_1),\dots,(p_N,q_N)\}\) be a bijection between indices of \(A'\) and \(B'\) chosen to minimize total distance. Then \[ \mathrm{SAMD}(A,B)=\frac{1}{N}\sum_{(i,j)\in M} \bigl[1-\cos(a_i,b_j)\bigr] \quad\text{where }|A'|=|B'|=N\text{ and }M\text{ is a one-to-one minimal matching.} \] The paper contrasts SAMD with two established LSCD metrics. Average Pairwise Distance (APD) computes \[ \frac{1}{m\cdot n}\sum_{a\in A}\sum_{b\in B}\delta(a,b), \] and is described as global, sensitive to bulk distribution shifts, and liable to dilute signal from small emerging senses. Average Minimum Distance (AMD) is directional, \[ \mathrm{AMD}(A\rightarrow B)=\frac{1}{|A|}\sum_{a\in A}\min_{b\in B}\delta(a,b), \] with symmetric AMD defined as \((\mathrm{AMD}(A\rightarrow B)+\mathrm{AMD}(B\rightarrow A))/2\); it is sensitive to local correspondences, but can map multiple points in \(A\) to the same point in \(B\) under many-to-one matching [2602.15716]. SAMD differs from AMD by enforcing one-to-one matching, thereby mitigating hubness, described as the case where a few points act as nearest neighbours for many. The paper states two reasons why symmetry matters: one-to-one alignment prevents over-representation of dominant “hub” embeddings in local-neighbour matches, and it guarantees the same number of matches in both periods and a truly symmetric notion of change. The method is also said to capture both distributional shifts and changes in concentration of usage clusters [2602.15716]. ## 3. Computation, hyperparameters, and complexity in LSCD The step-by-step computation of SAMD begins with two sets of embeddings \(A\) and \(B\), and a distance function \(\delta\), with cosine distance as the default [2602.15716]. The sets are equalized by setting \(N=\min(m,n)\); if \(m>N\), randomly sample \(N\) embeddings from \(A\), and if \(n>N\), sample \(N\) from \(B\). The resulting sets are \(A'\) and \(B'\). A pairwise distance matrix \(D\in\mathbb{R}^{N\times N}\) is then built with entries \(D_{ij}=\delta(a'_i,b'_j)\). The paper gives a greedy one-to-one matching procedure: initialize unmatched row and column sets, flatten all triples \((i,j,D_{ij})\), sort ascending by distance, and iterate through the sorted list, accepting a pair only if both its row and column remain unmatched. The normalized sum \(S/N\) is the SAMD score. As an alternative, one may solve the linear assignment problem via the Hungarian algorithm for the exact minimum total matching [2602.15716]. The design space includes several explicit choices. The distance function can be cosine distance or Euclidean distance, with the choice affecting sensitivity to vector norms. The sampling strategy can be random uniform down-sampling to size \(N\), or stratified sampling if there is sense imbalance. The matching algorithm can be greedy, with complexity \(O(N^2\log N)\) dominated by sorting all pairs, or the Hungarian algorithm with complexity \(O(N^3)\); greedy is described as faster and empirically near-identical. Dimensionality reduction may use full embeddings with \(D\approx 768\), PCA to \(K\) dimensions, definition-space projection, or random subspace. Optional outlier handling may clip distances above a threshold or remove outlier embeddings before matching; it is stated not to be required in practice, but may improve stability if a few usages are extreme [2602.15716]. The stated computational complexity uses \(N\) for the equalized number of embeddings per period and \(D\) for the embedding dimension. Pairwise distance computation costs \(O(N^2\cdot D)\), sorting all \(N^2\) distances costs \(O(N^2\log N)\), and the greedy matching pass costs \(O(N^2)\) in the worst case, for total complexity \(O(N^2\cdot(D+\log N))\). For very large \(N\) exceeding \(5\)k, the paper suggests approximate nearest neighbour structures to avoid the full distance matrix, further subsampling or clustering usages first, or using the Hungarian algorithm only on a top-\(k\) nearest-neighbour graph per point to reduce matrix size [2602.15716]. ## 4. Empirical behavior of SAMD in lexical semantic change benchmarks The LSCD study reports that, across multiple languages, encoder models, and representation spaces, AMD often provides more robust performance, particularly under dimensionality reduction and with non-specialised encoders, while SAMD excels with specialised encoders [2602.15716]. The paper’s broader conclusion is that LSCD may benefit from considering alternative semantic change metrics beyond APD and PRT, with AMD offering a robust option for contextualised embedding-based analysis. At the metric level, one reported result is that SAMD with PCA + XL-LEXEME achieves Spearman \(\rho\approx 0.694\), compared with APD at \(0.635\) and PRT at \(0.643\) on full embeddings. Under progressive PCA or random dimension reduction, SAMD is reported to maintain \(\rho\approx 0.40\) even at \(4\)–\(8\) dimensions, while APD and PRT collapse near \(0.0\)–\(0.3\). Across \(7\) languages and monolingual versus multilingual encoders, SAMD consistently outperforms or matches AMD, and both surpass APD and PRT, especially under dimensionality constraints [2602.15716]. The paper also isolates a representation-dependent distinction. In definition-space with \(K\approx 5\)–\(11\) dimensions, AMD slightly outperforms SAMD for non-specialised encoders, whereas SAMD excels with specialised ones due to its one-to-one constraint. This makes the one-to-one constraint central to the interpretation of the metric’s empirical profile, rather than a merely algorithmic detail [2602.15716]. Practical implementation guidance follows the same pattern. The workflow is: extract usage embeddings for each target word and time period using the chosen encoder; optionally apply PCA or a semantic projection; equalize sizes with \(N=\min(|A|,|B|)\); compute a distance matrix or KNN-filtered distances; run greedy matching or the Hungarian algorithm; and return the average matched distance. The paper’s implementation advice includes setting `max_samples` to control runtime on very large usage sets, using `sklearn.decomposition.PCA` to reduce dimension to \(K\), and pre-clustering usages and matching cluster centroids to reduce \(N\) for large-scale discovery [2602.15716]. ## 5. SAMD in coding theory: minimum average Hamming distance In coding theory, let \(A\subset\{-1,1\}^n\) be a binary code of size \(|A|=M\). Its average Hamming distance is \[ D(A)=\frac{1}{M^2}\sum_{x,x'\in A} d_H(x,x') =\sum_{i=0}^n i\,P^{(A)}(i), \] where \[ P^{(A)}(i)=\frac{\bigl|\{(x,x')\in A^2:d_H(x,x')=i\}\bigr|}{M^2}, \qquad i=0,1,\dots,n. \] The extremal quantity \[ \mathrm{SAMD}(n,M):=\min_{\substack{A\subset\{\pm1\}^n\\|A|=M}}D(A) \] is called the Symmetric Average Minimum Distance of a \((n,M)\) code [1910.09416]. A key fact in the Yu–Tan treatment is that \(D(A)\) can be re-expressed in terms of the dual distance distribution \(Q^{(A)}(k)\ge 0\), \(k=0,\dots,n\), satisfying \(\sum_k Q^{(A)}(k)=1/a\) with \(a=M/2^n\le \tfrac12\). The paper states \[ D(A)=\frac{n+1}{2}-\frac1{2a}+\frac12\sum_{k=2}^n Q^{(A)}(k), \] so minimizing \(D(A)\) is equivalent to minimizing \(\sum_{k=2}^n Q(k)\) under MacWilliams–Delsarte/Krawtchouk constraints [1910.09416]. Relaxing to an arbitrary nonnegative vector \(u=(u_0,\dots,u_n)\) yields a pair of finite-dimensional linear programs, identified as Fu–Wei–Yeung’s primal and dual LPs. The dual is written over variables \(x_k\), with nonnegativity constraints and inequalities involving \(K_k(1)-K_k(i)\), where \(K_k(i)\) is the \(k\)-th Krawtchouk polynomial in an \(n\)-cube. By strong duality, \(\Lambda(n;a)=\overline{\Lambda}(n;a)\), and hence for any code \(A\) of relative size \(a\), \[ D(A)\ge \frac12-\frac1{2a}+\frac12\,\overline{\Lambda}(n;a)+\frac n2. \] This reformulation makes SAMD a linear-programming extremal problem rather than a direct combinatorial search over codes [1910.09416]. ## 6. Improved coding-theoretic bounds, asymptotic optimality, and Fourier consequences Yu and Tan improve the coding-theoretic SAMD bound by exhibiting an explicit two-sparse dual feasible solution \[ x^*=\bigl(0,\dots,0,\;x^*_k,\;x^*_{k+1},\;0,\dots,0\bigr), \qquad k=2\left\lfloor\frac{\beta n}{2}\right\rfloor, \quad \beta\in\left(\tfrac12,1\right), \] whose only nonzero entries are chosen so that the endpoint constraints \(i=2\) and \(i=n\) tighten to equality [1910.09416]. The construction solves two linear equations in closed form, and the paper states that Krawtchouk-polynomial comparison lemmas on roots, monotonicity, and magnitude bounds imply that all other constraints are automatically satisfied for sufficiently large \(n\). Positivity of \(x^*_k\) and \(x^*_{k+1}\) holds when \(\beta>\tfrac12\) and \(n\gg 1\). Substituting \(x^*\) into the dual objective and letting \(n\to\infty\) yields \[ \liminf_{n\to\infty}\overline{\Lambda}(n;a) \ge \sup_{\tfrac12<\beta<1} -\frac{1+(1-2\beta)\bigl(\tfrac1a-1\bigr)} {2\beta(2\beta-1)} =\theta(a), \] with the closed-form piecewise function \[ \theta(a)= \begin{cases} \dfrac{(1-\sqrt a)^2}{a},&0