Lexical-Semantic Fusion

• Lexical-Semantic Fusion is a unified framework that combines symbolic lexical information with embedding-based semantic signals to improve robustness and expressiveness.
• It underpins diverse applications such as compositional semantics, information retrieval, and neural machine translation using fusion algorithms like CC-Fusion, RRF, and CLEAR.
• By merging formal typing systems with dense representations and hybrid evaluation metrics, the approach achieves greater accuracy, efficiency, and interpretability in language tasks.

Lexical-semantic fusion refers to a set of formal and algorithmic methodologies that integrate, align, or jointly leverage both lexical (symbolic, form-based, or type-theoretic) and semantic (distributional, embedding-based, or knowledge-driven) information in a unified representational and inferential framework. The motivations and implementations span compositional semantics, information retrieval, neural machine translation, linguistic resource construction, language modeling, and compositional ontologies, with the unifying principle that the fusion of lexical and semantic signals yields systems that are more robust, expressive, and effective than either component alone.

1. Formal Foundations and Typological Distinctions

Lexical-semantic fusion has its origins in the distinction between ontological concepts (entity types, selectional restrictions) and logical concepts (predicates, relations). In strongly-typed semantic frameworks, such as the type-theoretical models of Saba (Saba, 2018), Mery & Retoré (ΛTYₙ) (Mery et al., 2013), and the Montagovian Generative Lexicon (MGL) (Retoré, 2013), every linguistic variable is annotated with a type drawn from an ontologically motivated hierarchy:

• Ontological concepts: Represented as types, e.g., Human, Artifact, Event; serve as sortal constraints in grammar and composition.
• Logical concepts: Represented as predicates on said types, e.g., OLD(x:Human), DRIVE(x:Agent, y:Vehicle).

The fusion is achieved by:

• Assigning every word a principal λ-term and, where needed, a set of coercions or optional meaning-shifts, each annotated as flexible or rigid.
• Allowing context-driven type unification, with explicit rules for type inheritance, coercion, and the insertion of salient bridging relations (e.g., HASCONTENT, for copredication).
• Embedding the ontological type system natively in the semantic calculus (System F or similar), thereby treating compositional semantics and lexical adaptation uniformly.

This approach supports explicit resolution of metonymy, polysemy, copredication, missing-text phenomena, and fine-grained selectional restrictions via lexical–semantic fusion in the syntactic-semantic interface (Mery et al., 2013, Retoré, 2013, Saba, 2018).

2. Fusion Algorithms in Information Retrieval and Embedding Models

In IR, fusion refers to hybrid retrieval architectures that combine lexical exact-match (e.g., BM25, sparse vector models) with semantic similarity (e.g., dense embeddings from BERT or related architectures). State-of-the-art models demonstrate that:

• Convex Combination (CC) Fusion: Normalizes lexical and semantic scores per query (commonly via theoretical min-max) and composes them with a tunable weight $\lambda$:

  $$f_{\rm CC}(q, d) = \lambda\,\phi_{\rm Sem}(f_{\rm Sem}(q, d)) + (1-\lambda)\,\phi_{\rm Lex}(f_{\rm Lex}(q, d)),\quad 0\le\lambda\le 1$$

  This single-parameter model is robust to score normalization and is both sample- and domain-efficient, consistently outperforming reciprocal-rank-based fusions (Bruch et al., 2022).

• Reciprocal Rank Fusion (RRF): Combines the system ranks using a harmonic formula but is sensitive to its smoothing parameter and less robust across distributions and tasks (Bruch et al., 2022).
• Residual Embedding Fusion (CLEAR): Explicitly trains the neural embedding to focus on semantic signals not captured by BM25, reinforcing complementarity:

  $$s_{\rm CLR}(q, d) = s_{\rm lex}(q, d) + \alpha \langle \mathbf{r}_q, \mathbf{r}_d \rangle$$

  with a loss that adapts the margin based on lexical residuals (Gao et al., 2020).

• Dense Hybrid Representations (DHRs): Lexical scoring is densified (ex., via SPLADE or DeLADE), then concatenated or fused with semantic dense vectors in a gated inner-product system, enabling efficient retrieval with joint training (Lin et al., 2022).

Fusion consistently drives gains in recall, NDCG, robustness to domain shift, and efficiency, and enables minimal tuning for near-optimal performance even in low-resource or zero-shot settings (Bruch et al., 2022, Lin et al., 2022, Gao et al., 2020).

3. Fusion in Distributional and Lexical Semantic Resource Construction

Resource-centric fusion techniques combine high-coverage distributional representations of word senses (induced from corpus data via clustering or graph-based methods) with manually curated lexical-semantic ontologies (e.g., WordNet, BabelNet):

• Hybrid Aligned Resources (HAR): Begin with induced sense inventories, construct sense graphs, and align or type each sense in relation to the lexical ontology using context bag-of-words overlap and bootstrapped voting over hypernyms (Biemann et al., 2017).
• Lexical Chains with Embeddings: Construct flexible and fixed lexical chains using WordNet relations, reduce chains to representative synsets, and re-train embeddings (Chains2Vec) on these chains, resulting in multi-semantically fused document representations that significantly boost downstream classification (Ruas et al., 2021).

These methods achieve high mapping accuracy to lexical ontologies, dramatically improve word sense disambiguation performance, and facilitate unsupervised taxonomy induction, demonstrating the power of fusing corpus-driven senses with lexical networks (Biemann et al., 2017, Ruas et al., 2021).

4. Neural and Compositional Fusion in Language Generation and Modeling

Fusion schemes also enhance neural machine translation (NMT), language modeling (LM), and generation:

• Pointer-based Fusion for NMT: Integrates a symbolic bilingual lexicon with a neural decoder via gating at each timestep. Lexicon probabilities and neural outputs are blended to favor lexical matches for low-resource or out-of-vocabulary tokens, resulting in higher BLEU scores and greatly reduced parameter footprints (Gū et al., 2019).
• Sentence Embedding Fusion in LMs: Fixed-size semantic vectors (e.g., [CLS] embeddings from masked LLMs) are concatenated to LSTM inputs, allowing the autoregressive LM to condition on both token sequences and global semantic summaries, yielding reliable perplexity gains (Zouhar et al., 2022).
• Fuzzy-Membership Feature Fusion in Transformers: Parallel semantic channels encode interpretable, graded (fuzzy) token-level features—such as POS, polarity, and intensity—projected into hidden space and fused via a learned gating adapter. The semantic features may be softly or hard-conditioned for controllable language generation, delivering improved perplexity, out-of-domain generalization, and user-controllable outputs with minimal overhead (Huang et al., 14 Sep 2025).

These neural fusion paradigms enable modular and interpretable augmentation, improved generalization, and explicit semantic control in language modeling (Gū et al., 2019, Zouhar et al., 2022, Huang et al., 14 Sep 2025).

5. Lexical-Semantic Fusion in Evaluation and Systematic QA Metrics

Recent work in QA evaluation formalizes fusion at the metric level to overcome the deficiencies of both purely lexical (n-gram) and purely semantic (embedding-based) approaches:

• SMILE Metric: Computes a composite score that linearly fuses lexical exactness (exact match and n-gram embedding match) and sentence-level semantic similarity:

  $$s_{\mathrm{SMILE}}(y, y^*) = w\,\mathrm{SEM_{sent}}(y, \tilde y) + (1-w)\,\mathrm{LEX}(y, y^*)$$

  with $\tilde{y}$ a synthetic reference answer generated to match the style and verbosity of the prediction. Empirically, SMILE achieves the highest correlation with human judgments on diverse QA tasks and dramatically outperforms both standalone n-gram and embedding metrics (Kendre et al., 21 Nov 2025).

This class of metrics demonstrates that explicit balancing of lexical and semantic axes leads to higher-fidelity, interpretable, and computationally efficient evaluation on text, image, and video QA (Kendre et al., 21 Nov 2025).

6. Ontological Fusion and Semantic Web Representations

In ontological modeling and the semantic web, lexical-semantic fusion is realized by assigning complex, decomposed semantics directly to lexical entries at the class level:

• ILexicOn Architecture: Each interlingual lexical unit class (ILUc) is defined with OWL restrictions that encode its semantic decomposition, realized via "Conceptual Participant Slots" (ConP-slots) and enforced via meta-ontology (ILexiMOn). Data layer instantiations then inherit and fill these slots, embedding decompositions as class constraints (Lefrançois et al., 2012).
• Systematic ECD Compliance: Ensures explicitness, coherence, uniform inheritance, and compositional sufficiency, enabling the formal and operational bridging of explanatory dictionary traditions (ECD) with Semantic Web formalisms (Lefrançois et al., 2012).

This explicit projection of semantic decompositions into the class layer achieves true lexical-semantic fusion, making ontology classes self-sufficient semantic frames (Lefrançois et al., 2012).

7. Methodological, Cognitive, and Practical Implications

The various strands of lexical-semantic fusion intersect across the following axes:

Domain	Fusion Mechanism	Impact/Advantages
Formal semantics	Typed λ-calculus, coercions, ontology-based sort systems	Fine-grained selection, copredication, pragmatics
Embedding/IR	Score fusion, dense hybrid representations, joint training	Recall/NDCG gains, robustness, efficient retrieval
Resource creation	Graph alignment, chain-based embedding induction	Expanded coverage, unsupervised sense induction
Neural LM/NMT	Gated/concatenative fusion, pointer-based gating	Lower perplexity/BLEU, controllable generation
Evaluation metrics	Weighted composite metrics (SMILE)	Closer human alignment, nuance in scoring
Ontology/SemWeb	Class-level semantic projections, OWL-based constraints	Uniform, explicit, and extensible semantic frames

The effect of lexical-semantic fusion is to systematically overcome limitations of either symbolic or distributional methods in isolation, supporting modularity, explainability, generalization, and domain adaptation. A plausible implication is that fusion architectures will become increasingly standard across all levels of computational semantics, with efficient, differentiable, and interpretable mechanisms for joint leveraging of symbolic and statistical signals.

References:

(Bruch et al., 2022, Giunchiglia et al., 2022, Saba, 2018, Biemann et al., 2017, Gū et al., 2019, Cotterell et al., 2017, Gao et al., 2020, Lin et al., 2022, Lefrançois et al., 2012, Huang et al., 14 Sep 2025, Liu et al., 2020, Mery et al., 2013, Ruas et al., 2021, Kendre et al., 21 Nov 2025, Mery et al., 2013, Retoré, 2013, Zouhar et al., 2022)