SWORDS: Lexical Substitution in Context

• SWORDS is a framework for context-aware lexical substitution that replaces target words with contextually appropriate synonyms while preserving meaning and syntax.
• It benchmarks substitute quality by scoring candidates on grammaticality, semantic fit, and acceptability using graded human judgments and predefined thresholds.
• Innovative techniques like context augmentation and multi-signal ranking significantly enhance performance, making SWORDS robust across multiple languages and domains.

Searching arXiv for recent and foundational work on SWORDS, lexical substitution, and context-aware synonym replacement. “Substituting WORDs with Synonyms (SWORDS)” denotes a family of problems and methods concerned with replacing a target word with an alternative that preserves meaning under contextual constraints. In contemporary NLP, the term is closely associated with lexical substitution: given a context $c$ and a target word $w$, a system must return substitutes $w'$ that fit the specific sentence rather than merely appearing in a context-free thesaurus (Lee et al., 2021). The label also refers to two concrete research artifacts: SWORDS, the Stanford Word Substitution Benchmark for lexical substitution (Lee et al., 2021), and SWORDS, the code-switching algorithm introduced in UniCoM, where selected words are replaced with translation-equivalent words from another language while preserving sentence semantics and matrix-language word order (Lee et al., 21 Aug 2025). Across these uses, the central issue is the same: synonym substitution is only reliable when sense, syntax, usage, and broader contextual relevance are explicitly modeled (Vladika et al., 6 Feb 2025).

1. Task definition and conceptual scope

Lexical substitution is formalized as the task of mapping a context $c$ and a target word $w$ to a ranked list of substitutes $w'$ that can replace $w$ in $c$ without breaking meaning or acceptability (Lee et al., 2021). In SWORDS, each triple $(c,w,w')$ carries a score equal to the fraction of annotators who would actually consider using $w'$ to replace $w$0 as the author of the sentence, turning lexical substitution into a graded contextual appropriateness problem rather than an exercise in dictionary lookup (Lee et al., 2021).

A central clarification in recent work is that lexical substitution is not identical to synonym substitution. “Lexical Substitution is not Synonym Substitution” argues that the substitute should not necessarily be only synonymous, but should also fit well into the surrounding context of the target word while preserving the sentence’s grammatical structure (Vladika et al., 6 Feb 2025). This distinction is reinforced by the SWORDS benchmark paper, which contrasts context-sensitive substitution with context-free thesaurus relations and positions lexical substitution as a task for writing assistance rather than as a mere synonym lookup problem (Lee et al., 2021).

This suggests a useful technical distinction between lexical relatedness and substitutability. A word may be semantically related, or even listed as a synonym in a lexical resource, yet still be contextually inappropriate, stylistically odd, or syntactically ill-formed in a given sentence (Lee et al., 2021). The adversarial-attack literature makes the same point negatively: many purported “synonym substitution attacks” rely on replacements that are ungrammatical or do not preserve the original sentence’s semantics, showing that surface-level synonymy claims are often insufficient (Chiang et al., 2022).

2. Benchmarking with SWORDS

SWORDS was introduced to address two limitations of earlier lexical substitution benchmarks: low substitute coverage and weak quality control (Lee et al., 2021). Existing resources such as SemEval-2007 and CoInCo relied on human recall as the only source of substitutes, which underrepresents the alternatives that are hardest for humans to think of and most useful for writing assistance (Lee et al., 2021).

The benchmark reframes lexical substitution as a classification task over a predefined candidate set. For each $w$1 pair, candidates are drawn from Roget’s Thesaurus for the target part of speech and augmented with human-generated substitutes from CoInCo; annotators then judge each triple $w$2 with a binary decision indicating whether they would actually consider using the substitute as the author of the sentence (Lee et al., 2021). The final score is the fraction of positive labels, usually out of 10 judgments, and operational thresholds define acceptable substitutes as score $w$3, conceivable substitutes as score $w$4, and inconceivable substitutes as score $w$5 (Lee et al., 2021).

The resulting dataset contains 1,132 contexts, 1,132 targets, 68,683 unique substitutes, and 375,855 total labels (Lee et al., 2021). It averages 60.7 substitutes per target, including 21.4 conceivable and 4.1 acceptable substitutes, and therefore has 4.1x more conceivable substitutes per target and 1.8x more acceptable substitutes per target than the overlapping CoInCo estimate (Lee et al., 2021). When equalizing the number of substitutes, SWORDS yields an average score of 50.7% versus 34.4% for CoInCo relabelings, which the benchmark authors summarize as 1.5x higher average appropriateness (Lee et al., 2021).

The benchmark also makes explicit that substitutes may be words or phrases and are represented as lemmas, so inflection is deliberately separated from contextual appropriateness (Lee et al., 2021). This broad coverage creates a stricter evaluation setting for systems that must retrieve not just obvious substitutes but also plausible, less recalled alternatives.

3. Context-aware modeling of substitutes

Recent systems treat contextual fit as the core modeling problem. An unsupervised lexical simplification and substitution method with context augmentation generates candidates by combining contextual–decontextual embedding similarity with substitutes proposed by T5 or mT5 on additional sampled contexts of the target word (Wada et al., 2023). Given a target word $w$6 in context $w$7, the method first computes a contextualized embedding $w$8, constructs multi-prototype decontextualized embeddings $w$9 for vocabulary items, and scores candidates by

$w'$0

with an additional fastText similarity term when subword segmentation is noisy (Wada et al., 2023).

Its main novelty is context augmentation. For the target word $w'$1, the method samples 300 sentences containing $w'$2 from monolingual corpora, clusters their contextualized embeddings with $w'$3-means using $w'$4, masks $w'$5 in those sampled sentences, and uses T5 or mT5 to generate single-word substitutes (Wada et al., 2023). Cluster weights $w'$6 are defined by overlap with the top similarity-based candidates from the original target context, and the augmented-context score is

$w'$7

where $w'$8 indicates whether T5 or mT5 generated $w'$9 for the masked sentence $c$0 (Wada et al., 2023).

Candidates are then reranked by four signals: embedding similarity, LM perplexity or generation probability, word frequency, and the augmented-context score $c$1 (Wada et al., 2023). Weighted rank aggregation produces the final list, and for lexical substitution on SWORDS the simplicity term is disabled and the candidate pools are enlarged to $c$2 and $c$3 (Wada et al., 2023). The method achieves a state-of-the-art result on the SWORDS lexical substitution dataset and substantially outperforms other unsupervised systems across English, Portuguese, and Spanish on TSAR-2022 (Wada et al., 2023).

Other context-aware architectures integrate external lexical knowledge. LexSubCon combines a BERT proposal score based on a mix-up embedding of the target, gloss similarity from WordNet definitions selected by GlossBERT, sentence-level semantic similarity computed with a fine-tuned sentence similarity model, and a candidate validation score based on contextual impact (Michalopoulos et al., 2021). Its final score is a linear combination

$c$4

with $c$5, $c$6, $c$7, and $c$8 (Michalopoulos et al., 2021). This architecture exemplifies a broader trend: contextual substitution quality improves when local fit, sense-level similarity, and whole-sentence semantic preservation are modeled jointly.

4. Sense discrimination, lexical resources, and synonym structure

Sense awareness is a prerequisite for reliable substitution. Resource-oriented work has long emphasized that a synonym is only safe when the active sense is known. A WSD-specific WordNet organizes senses of polysemous words by clue words rather than by broad synsets and hypernym trees, precisely to avoid “ambiguity in ambiguity” caused by non-discriminative higher-level relations (Dhungana et al., 2014). In that model, each sense $c$9 is associated with a clue-word collection $w$0, and overlap with context words supports sense selection before any synonym choice is made (Dhungana et al., 2014).

Vector-based WSD follows a similar logic. An $w$1-filtration method for Russian defines synsets $w$2 for a target word $w$3, filters both sentence and synset vectors by cosine thresholds, and chooses the synset with maximal set-level proximity $w$4 (Kirillov et al., 2018). The practical implication is that synonyms should be selected from the synset corresponding to the resolved sense rather than from an undifferentiated synonym list (Kirillov et al., 2018).

Graph-based synset induction addresses the same problem at resource-construction time. Watset builds a synonym graph, applies local clustering to ego networks to induce sense-specific nodes, disambiguates neighbors, and then performs global clustering to obtain synsets (Ustalov et al., 2017). Its local-global meta-clustering yields fuzzy clustering at the word level while keeping individual sense nodes in hard clusters, which is exactly the structure needed for sense-aware lexical substitution (Ustalov et al., 2017).

Theoretical work further sharpens the distinction between word-level relatedness and sense-level identity. “Synonymy = Translational Equivalence” treats synsets as equivalence classes of absolute synonymy of senses and multi-synsets as equivalence classes of semantic equivalence across languages (Hauer et al., 2020). Under that view, word-level substitution is valid when the relevant senses belong to the same synset; near-synonymy at the word level is insufficient unless the active sense is fixed (Hauer et al., 2020).

5. Limits of naive synonym substitution

A recurrent finding across several lines of work is that naive synonym replacement is unreliable. In adversarial NLP, four widely used substitution methods generate large fractions of invalid substitution words that are ungrammatical or do not preserve the original sentence’s semantics (Chiang et al., 2022). For PWWS on AG-News, only 20.2% of 26,600 substitutions were matched-sense synonyms, whereas 75.4% were mismatched-sense synonyms and about 3.8% were morphological substitutions (Chiang et al., 2022). For counter-fitted GloVe $w$5-nearest neighbors and MLM-based generators, a 30-word candidate set contains far fewer than one true context-appropriate synonym on average, with most candidates falling into an “others” category rather than matched-sense synonymy (Chiang et al., 2022).

Embedding-based and sentence-embedding constraints only partly solve this. Word embedding cosine similarity can filter random unrelated words and antonyms, but it cannot reliably separate matched-sense from mismatched-sense synonyms and often overfavours morphological variants that are ungrammatical in context (Chiang et al., 2022). This result complements the benchmark-side critique in SWORDS, where many CoInCo substitutes become unacceptable under stricter contextual judgments and benchmark scores are shown to reflect ease of recall rather than appropriateness (Lee et al., 2021).

Recent lexical substitution work reaches a similar conclusion from a generation perspective. ConCat, which concatenates a masked sentence and the original sentence as input to RoBERTa, is motivated by the observation that standard masked-language-model substitution either overfits to the target word or produces context-fitting words that drift semantically (Vladika et al., 6 Feb 2025). On LS07, CoInCo, and SWORDS, ConCat improves over a recreated dropout baseline on best, best-mode, oot, oot-mode, $w$6, and $w$7, and human evaluation shows that respondents prefer its substitutes overall (Vladika et al., 6 Feb 2025). The paper also documents potential pitfalls in CoInCo, including context-poor sentences, noisy labels, and substitutes that are inappropriate in context (Vladika et al., 6 Feb 2025).

A plausible implication is that “synonym substitution” is best treated as a constrained inference problem over sense, context, syntax, and usage, not as a lexical lookup problem. The benchmark, generation, and adversarial literatures all converge on this point, even though they operationalize it differently (Lee et al., 2021, Vladika et al., 6 Feb 2025, Chiang et al., 2022).

6. Applications beyond benchmarked lexical substitution

The same substitution machinery appears in domains outside writing assistance. In robustness research, synonym replacement is used both to attack and to defend classifiers. Randomized Substitution and Vote treats synonym-substitution attacks as optimized replacement sequences and detects adversarial examples by randomly substituting words with synonyms, generating $w$8 perturbed texts, aggregating logits, and flagging disagreement between the original prediction and the voted prediction (Wang et al., 2021). Because the method requires no architecture modification or extra training, it treats synonym substitution as a probe of local decision-boundary fragility (Wang et al., 2021).

A complementary defense, the Synonym Encoding Method, clusters synonyms in counter-fitted embedding space and maps each cluster to a unique code word before the classifier’s input layer (Wang et al., 2019). Its encoder $w$9 is intended to make many nearby synonym variants collapse to the same representation, so that for many $w'$0, $w'$1 (Wang et al., 2019). Here synonym substitution is not used to produce alternatives for humans but to smooth model behavior against word-level perturbations.

In text-to-SQL, synonym substitution exposes the reliance of schema linking on surface lexical overlap. Spider-Syn replaces schema-related words with manually selected synonyms that reflect real-world paraphrases, and exact-match accuracy drops sharply for GNN, IRNet, RAT-SQL, and RAT-SQL$w'$2 when explicit correspondence between questions and schema names is removed (Gan et al., 2021). Manual and automatic schema synonym annotations partly restore performance, showing that substitution robustness depends on explicitly representing lexical alternatives (Gan et al., 2021).

The term SWORDS is also used in multilingual speech generation. In UniCoM, SWORDS creates intra-sentential code-switching speech by replacing selected words in a matrix-language sentence with semantically equivalent words from an embedded language while conditioning on part of speech and preserving original sentence semantics and language-specific word order (Lee et al., 21 Aug 2025). Word-level translation pairs are extracted by GPT-4o-mini, aligned to speech segments with MMS-FA, and substituted in both text and audio; in CS-FLEURS, substitution is limited to nouns, verbs, and interjections, with at most three substitutions per sentence (Lee et al., 21 Aug 2025). This is not lexical substitution in the benchmark sense, but it preserves the same operational core: meaning-preserving replacement under contextual and structural constraints.

7. Resource enrichment and future directions

Reliable substitution systems depend on the quality of lexical resources. “Grouping Synonyms by Definitions” groups French synonyms by dictionary senses by comparing definition indices from TLFi and synonym dictionaries, reaching at best precision 67% and recall 71% against a lexicographer-annotated gold standard (0909.3445). The work shows that dictionary-based sense partitioning is feasible and that gloss overlap remains competitive for synonym-to-sense assignment (0909.3445).

Arabic resource construction has moved toward graded synonymy. A benchmark and scoring algorithm for enriching Arabic synonyms assigns candidates fuzzy values in $w'$3 using cyclic paths in synonym and translation graphs and evaluates agreement against four linguists with RMSE and MAE (Ghanem et al., 2023). The scoring function

$w'$4

models synonymy strength rather than binary membership, which is directly relevant to ranked substitution systems that need thresholds or graded preferences rather than categorical synonym labels (Ghanem et al., 2023).

Sense-aware embeddings provide another avenue. The SWSDS model for Chinese first applies SememeWSD to label a polysemous target with a sense ID, then retrieves the top 10 synonyms of that sense from OpenHowNet and averages their vectors to form a sense embedding (Zhou et al., 2022). On LCQMC, this improves semantic similarity accuracy from 67.9% to 71.9%, suggesting that synonym sets can serve as robust semantic representatives once sense has been resolved (Zhou et al., 2022).

At the same time, distributional work on near synonyms cautions against assuming full interchangeability. A Random Forest trained on Hindi word embeddings can distinguish Sanskrit-origin from Perso-Arabic-origin near synonyms with average accuracy of about 88%, and around 95% after removing the most error-prone words, indicating that usage patterns preserve etymological and cultural signals even when words share dictionary-level meaning (Bąkowski, 1 Apr 2026). This suggests that contextual appropriateness is shaped not only by denotation and syntax but also by register, cultural association, and historical lexical strata.

Taken together, these findings define the current research frontier. High-quality SWORDS systems require broad candidate coverage, explicit sense discrimination, context-sensitive ranking, and lexical resources that encode more than coarse synonymy (Lee et al., 2021, Wada et al., 2023, Ghanem et al., 2023). They also require evaluation protocols that do not confuse easy recall with appropriateness and do not treat lexical substitution as a synonym lookup task by default (Lee et al., 2021, Vladika et al., 6 Feb 2025).