Semantic Stability (SS) Explained

• Semantic Stability (SS) is the persistence and invariance of meaning relationships across dynamic linguistic and computational contexts.
• Researchers quantify SS with domain-specific metrics like RBO, PC@k, and embedding drift to compare systems in social tagging, LLMs, and recommenders.
• High SS underpins model reliability, enhances recommender performance, and guides our understanding of linguistic evolution despite perturbations.

Semantic stability (SS) is a multidimensional concept denoting the persistence, robustness, or invariance of meaning relationships under change—across crowds, time, paraphrase, or perturbation. The term manifests across computational linguistics, evolutionary linguistics, cognitive neuroscience, LLM evaluation, embedding architectures, and recommender systems, yet is defined and operationalized differently within each domain. Across these lines of research, SS is consistently treated as a property reflecting consensus, structural invariance, or diagnostic reliability, measured by formal metrics linked to rank lists, embedding drift, response invariance, or phylogenetic rates.

1. Definitions and Theoretical Foundations

SS refers, in its most abstract sense, to the degree to which semantic representations, descriptors, or behaviors resist transformation in the face of new usage, rewording, or system modification:

• In social tagging systems, SS is achieved when a resource’s tag distribution converges: both the set of tags and their rank order become stable over time and user crowds (Wagner et al., 2013). Instability implies an unresolved contest over descriptors.
• In LLMs, SS is the model’s tendency to generate invariant outputs when exposed to paraphrastic (meaning-preserving) input perturbations; instability reveals “hallucination” as variance-driven divergence, distinct from bias or calibration failure (&&&1&&&, Li et al., 11 Jun 2025).
• In recommendation systems, SS refers to the temporal invariance of item embedding vectors in the face of identity churn, hash collisions, or parameterization drift (Zheng et al., 2 Apr 2025).
• In historical linguistics and cognitive science, SS measures the resistance of meaning categories to replacement or reorganization over evolutionary time, or the cross-cultural invariance of lexical-semantic network axes (Bowern, 2019, Ploux et al., 2017).

The unifying principle is that high SS evidences persistent, interpretable, and reliable structures—whether in crowd tags, neural responses, LLM outputs, or item embeddings—despite perturbation or temporal processes.

2. Formalization and Metrics

SS is operationalized differently across domains using rigorously defined, domain-specific metrics.

• Social tagging: Principal metric is Rank-Biased Overlap (RBO) between ranked tag lists at successive time points, with stabilization defined when RBO exceeds a threshold $k$, and system-level SS being the fraction of resources stabilized at $(t,k)$, denoted $f(t,k)$ (Wagner et al., 2013).
• LLM structural invariance:
  • Paraphrase Consistency (PC@k): Given $k$ paraphrases $x_1,\dots,x_k$ of a prompt $x$, and their greedy-decoded outputs $a_1,\dots,a_k$, $PC@k(x) = \frac{1}{k} \max_a |\{i : a_i = a\}|$, with global SS as $\mathbb{E}_x[PC@k(x)]$ (Flouro et al., 11 Jan 2026).
  • Prompt-Based Semantic Shift (PBSS): For paraphrase set $\mathcal{P}$ and model $f$, PBSS between prompts $p_i, p_j$ is $D(p_i,p_j) = 1 - \cos( s(f(p_i)), s(f(p_j)) )$, where $s$ is a sentence embedding; SS is summarized by CDFs or mean/max PBSS across paraphrases (Li et al., 11 Jun 2025).
• Embedding Drift in Recommender Systems:
  • $L_2$-drift: $$\Delta_{L2} = \mathbb{E}_{i,t}\big[\| \mathbf{e}_i^t - \mathbf{e}_i^{t+1} \|_2^2 \big]$$.
  • Cosine-variance: $$\mathrm{Var}_{\cos} = \mathrm{Var}_{i,t}\left[ \cos(\mathbf{e}_i^t, \mathbf{e}_i^{t+1}) \right]$$.
  • Lower drift/variance implies higher SS (Zheng et al., 2 Apr 2025).
• Lexical Semantic and Evolutionary Phylogenetics:
  • Embedding-based shift: $$\Delta_{embed}(w; t_1, t_2) = 1 - \cos( E(w, t_1), E(w, t_2) )$$.
  • Phylogenetic stability: $SS_m \approx 1/\sigma^2_m$ with $\sigma^2$ as the replacement rate, or $SS_m = \exp(-R_m)$ where $R_m$ is the Markov-process state-change rate (Bowern, 2019).
  • Structural stability (CA inertia and dispersion): Proportion of inertia explained by major axes in correspondence analysis, stability of category factor scores across languages or modalities, comparative dispersion metrics (Ploux et al., 2017).

3. Experimental Designs and Methodological Frameworks

Social Tagging and Consensus Formation

Social tagging SS studies measure how user-generated tag streams on platforms (e.g., Delicious, LibraryThing) stabilize over time using RBO and system-level f(t,k) curves. Datasets sample heavily and moderately tagged resources; time-series approaches monitor temporal evolution of tag rankings. Controls include random stream simulations and natural-language analogs (Wagner et al., 2013).

LLM Paraphrase Robustness and Behavioral Drift

LLM studies generate controlled paraphrase sets (via templated rules or auxiliary models), then measure response (in)variance under deterministic (temperature=0) decoding. PBSS evaluates all pairs in paraphrase sets, summarizing drift via mean/max distances and empirical CDFs. Experimental sweeps compare model architectures, alignment levels, tokenization strategies, and decoding temperatures (Li et al., 11 Jun 2025, Flouro et al., 11 Jan 2026).

Embedding Representation Stability in Recommenders

Large-scale recommender architectures define stability as temporal invariance of item embeddings under ID churn. Semantic ID and prefix n-gram clustering construct semantically meaningful embedding collisions, quantified by longitudinal $L_2$ and cosine-drift metrics. Production pipelines are evaluated by normalized-entropy (NE) changes and variance in A/B test prediction scores (Zheng et al., 2 Apr 2025).

Cross-Language and Evolutionary Semantic Structures

Neuroscientific and phylogenetic SS studies apply correspondence analysis (CA) to ERP or corpus-based matrices. Stability indices include explained inertia, dispersion, and prototypicality of semantic categories. Cross-linguistic reproducibility is established via axis correlations, dispersion metrics, and replicated structural factors (e.g., living/nonliving, person-centered gradients) (Ploux et al., 2017, Bowern, 2019).

4. Empirical Findings and Key Outcomes

Social Tagging Streams

Resources in Delicious and LibraryThing reach high SS rapidly: over 90% stabilize to RBO > 0.7 after ~1,000 assignments. Twitter lists are slower/less stable (RBO ≈ 0.6), while random streams never approach meaningful stability; natural-language controls stabilize only modestly. SS is maximized when imitation dynamics are combined with a nonzero rate of new tag injection from shared background knowledge (optimal at ~70% imitation), as pure imitation or pure background alone yields inferior convergence (Wagner et al., 2013).

LLM Instability under Paraphrase

Dense LLMs (Qwen3-0.6B) show low paraphrase agreement (SS ≈ 24%), but systematic sparsity reduction raises SS (peak ≈ 56% at 32% sparsity) before bias collapse reduces it again; excessive pruning tips models into bias-dominated high-agreement-on-wrongness (Flouro et al., 11 Jan 2026). PBSS drift magnitudes reveal strong model tiering: high-capacity/tuned models show sharply higher consistency under rewording than lightly tuned or legacy architectures. Decoding temperature adjusts variance but preserves these rank orderings (Li et al., 11 Jun 2025).

Embedding and Recommendation Stability

Semantic ID with prefix n-gram parameterization reduces embedding drift—both $L_2$ and cosine-based—especially for infrequent/tail IDs and under high churn. Gains translate to improved normalized-entropy, tail recall, overfitting mitigation, and lower prediction-score variance in live deployment (Zheng et al., 2 Apr 2025).

Cognitive and Phylogenetic Invariants

Semantic spaces, as revealed by CA of ERP and corpus data, show that major axes (living/nonliving, prototypicality of animals, person-centric gradients) are robustly conserved across Chinese and French. High SS is thus associated with invariant dimensions of cognitive lexicon organization, supported both by neurophysiological networks and corpus-extracted semantic structures (Ploux et al., 2017). In phylogenetic models, meaning classes such as pronouns and body parts exhibit markedly low replacement rates, while high-turnover classes (e.g., color terms, technology) show instability. SS is maximized for categories with moderate synonymy and low rates of loans and singletons (Bowern, 2019).

5. Causal Factors and Theoretical Insights

Social Tagging

Imitation, as modeled by Polya urn processes, amplifies convergence only when modulated by injection of tags from shared knowledge bases; pure imitation is insufficient for stabilization, as it prevents the entrance of new, consensus-building descriptors.

LLMs

Variance in LLM outputs is increased by redundant or unstable internal pathways, tokenization stochasticity, and low-level decoding sensitivity. Alignment fine-tuning suppresses token-level drift, while sparsity pruning selectively removes unstable predictive “modes” (analogous to noise reduction in PCA/SVD), increasing SS until representational bias dominates (Flouro et al., 11 Jan 2026, Li et al., 11 Jun 2025).

Embedding Stability

Random hash parameterizations induce unstructured collisions, degrading SS through contradictory gradient updates, especially under item identity churn. Semantic ID and prefix n-gram parameterization realize structured embedding sharing, reinforcing similar items and minimizing drift (Zheng et al., 2 Apr 2025).

Lexical and Evolutionary Semantics

Semantic categories with minimal synonymy/loan rates and persistent community-level consensus exhibit slowest rates of change. Structural axes corresponding to deeply grounded ontological distinctions show highest cross-lingual and modal SS (Ploux et al., 2017, Bowern, 2019).

6. Implications, Limitations, and Future Directions

SS serves as a foundational diagnostic axis, complementing accuracy and calibration in model and system evaluation. It is crucial in agentic AI contexts where semantic variance under perturbation can cascade into functional unreliability. In LLMs, protocols for SS evaluation are increasingly mandated for regulatory compliance (e.g., EU AI Act). In recommenders, embedding SS directly impacts downstream prediction reliability.

Methodological limitations include sensitivity of RBO to parameter $p$ (though results stabilize for $p\ge0.5$), the use of global rather than resource-specific knowledge in simulation, and by-design focus on system-level rather than per-user stability (Wagner et al., 2013). In LLMs, prompt canonicalization, tokenization regularization, and consistency-driven fine-tuning are recommended to mitigate drift (see PBSS protocol suggestions) (Li et al., 11 Jun 2025). In the embedding domain, further work is needed on dynamically evolving semantic IDs attuned to item and user histories (Zheng et al., 2 Apr 2025). In cognitive and evolutionary linguistics, optimal signal is captured by inclusion of meaning classes that are neither strictly invariant nor overly labile (Bowern, 2019).

7. Cross-Domain Synthesis

The notion of semantic stability—while instantiated in diverse mathematical tools, datasets, and operational settings—serves a universal function: surfacing reliable, interpretable, and robust meaning structures in distributed, evolving, or stochastic systems. Whether manifesting as consensus over tags, response invariance under paraphrase, embedding resilience under churn, or phylogenetically persistent meanings, high SS is signal of underlying regularity and communicative functionality. Its rigorous measurement—using RBO, PC@k, PBSS, $L_2$-drift, phylogenetic rates, or CA inertia—enables comparative system assessment, reveals cascades of instability, and guides the design of more dependable linguistic, recommender, and AI architectures [(Wagner et al., 2013); (Ploux et al., 2017); (Li et al., 11 Jun 2025); (Flouro et al., 11 Jan 2026); (Zheng et al., 2 Apr 2025); (Bowern, 2019)].