Semantic Similarity in a Taxonomy: An Information-Based Measure and its Application to Problems of Ambiguity in Natural Language (1105.5444v1)

Abstract: This article presents a measure of semantic similarity in an IS-A taxonomy based on the notion of shared information content. Experimental evaluation against a benchmark set of human similarity judgments demonstrates that the measure performs better than the traditional edge-counting approach. The article presents algorithms that take advantage of taxonomic similarity in resolving syntactic and semantic ambiguity, along with experimental results demonstrating their effectiveness.

Semantic Similarity in a Taxonomy: An Information-Based Measure and its Application to Problems of Ambiguity in Natural Language

Philip Resnik's paper presents an approach to measure semantic similarity within an "is-a" taxonomy by leveraging the concept of shared information content. This method addresses the limitations of traditional edge-counting techniques, which often suffer from variability in taxonomic link distances. The efficacy of the proposed measure is demonstrated by rigorous experimental evaluations against human similarity judgments and its application to resolving syntactic and semantic ambiguities in NLP.

Measurement Methodology

The core innovation of Resnik's measure lies in its use of information content. By associating probabilities with concepts in a taxonomy, the measure quantifies semantic similarity based on the amount of shared information:

$\text{sim}(c_1, c_2) = -\log p(c)$

where $p(c)$ is the probability of encountering an instance of the concept $c$, which subsumes both $c_1$ and $c_2$.

Empirical Evaluation

The measure was subjected to rigorous testing using human similarity judgments as a benchmark. The evaluation employed:

• A replication experiment: Human subjects rated noun pairs from Miller and Charles (1991), revealing a strong correlation ($r = 0.885$) between human judgments and the proposed measure.
• Comparative analysis: The information-based measure significantly outperformed traditional edge-counting approaches, with a correlation of $r = 0.79$ versus $r = 0.55$ for edge counting.

Applications in NLP

Coordination Ambiguity

Resnik explores the application of the measure to resolve coordination ambiguity in noun phrases. In syntactic constructions like "n1 and n2 n3," the method discerned the correct grouping based on semantic similarity and syntactic agreement:

• First Experiment: Applied to noun phrases extracted from the Penn Treebank corpus, the method showed a $49.4\%$ accuracy rate based on number agreement alone, and higher coverage when combined with other strategies.
• Second Experiment: Improved by combining weighted semantic similarity with selectional association measures, achieving $67.43\%$ accuracy.

Word Sense Disambiguation

The measure was also employed for word sense selection in contexts where nouns appeared in synonymous groupings:

• Algorithm Development: WordNet senses were identified using an algorithm that assigns confidence values based on shared information content among noun groupings.
• Evaluation: Applied to the CETA Chinese-English dictionary, the algorithm demonstrated usefulness in filtering irrelevant senses and guiding human annotators in semi-automated settings.

Comparative Analysis with Related Work

Resnik's measure stands out in comparison to other semantic similarity measures:

• Leacock and Chodorow (1998): Proposed a path-length-based measure. Though initially similar in performance, follow-up experiments demonstrated that Resnik's measure was more effective in avoiding spurious similarities.
• Lin (1998): Introduced an alternative information-theoretic measure normalized by the combined information content of the concepts. While Lin's measure presents a theoretically elegant formulation, the evaluation indicates Resnik's approach maintains a higher correlation with human judgments.

Practical Implications and Future Work

The application of taxonomic similarity has significant practical implications for various NLP tasks, including machine translation, information retrieval, and lexical resource development. The measure's deployment in real-world applications, such as linking the Wordsmyth English Dictionary-Thesaurus (WEDT) to WordNet, illustrates its utility in enhancing digital lexical resources.

Future research could explore refining these similarity measures and exploring their integration into more complex NLP systems. Further experimentation across diverse datasets and linguistic contexts will be crucial to validate and extend the applicability of these methodologies.

By demonstrating that semantic similarity as measured by shared information content can outperform traditional methods, Resnik's work offers a valuable contribution to the computational linguistics field, opening avenues for more nuanced language understanding and processing.