A Survey on Recent Advances in Named Entity Recognition from Deep Learning models (1910.11470v1)

Abstract: Named Entity Recognition (NER) is a key component in NLP systems for question answering, information retrieval, relation extraction, etc. NER systems have been studied and developed widely for decades, but accurate systems using deep neural networks (NN) have only been introduced in the last few years. We present a comprehensive survey of deep neural network architectures for NER, and contrast them with previous approaches to NER based on feature engineering and other supervised or semi-supervised learning algorithms. Our results highlight the improvements achieved by neural networks, and show how incorporating some of the lessons learned from past work on feature-based NER systems can yield further improvements.

Advances in Named Entity Recognition via Deep Learning Architectures

This paper provides an extensive survey of developments in Named Entity Recognition (NER) with a focus on deep learning models. NER is a significant task within NLP, crucial for applications such as question answering and information retrieval. The paper contrasts recent neural approaches to NER against traditional feature-engineered systems, offering a nuanced understanding of their respective advancements and highlighting areas where neural networks outperform previous methodologies.

Key Findings

The survey categorizes NER systems into several groups: knowledge-based, unsupervised, bootstrapped, feature-engineered, and feature-inferring neural models. Each category is explored with respect to its methodology and impact on NER performance across various datasets and languages.

1. Feature-engineered Systems: Historically, NER systems relied heavily on handcrafted features and gazetteer resources. Systems such as Hidden Markov Models (HMM), Support Vector Machines (SVM), and Conditional Random Fields (CRF) were prevalent, leveraging orthographic and linguistic features to improve recall and precision. However, these approaches often required extensive domain-specific knowledge.
2. Deep Learning Architectures: Recent neural network models have demonstrated superior performance over traditional systems. The paper highlights several architectures:
   • Word-Level Models: Bi-LSTM and Convolutional Neural Networks (CNN) architectures using word embeddings are discussed, showcasing their effectiveness with minimal feature engineering.
   • Character-Level Models: These models focus on character sequences, enabling robust handling of out-of-vocabulary words and morphological variations.
   • Hybrid Word+Character Models: Combining word embeddings with character-level RNNs, these models capture both global context and sub-word information, resulting in substantial performance gains.
3. Model Performance: Neural network systems consistently surpass feature-engineered models in various datasets, with word+character models generally achieving the highest accuracy. Notably, the integration of affix embeddings into these models further enhances their performance, as evidenced by improvements across multiple languages in the CoNLL and DrugNER datasets.

Implications and Future Directions

The survey underscores the shift towards neural models in NER, primarily due to their adaptability and reduced reliance on handcrafted features. This advancement suggests a broader trend in NLP towards models that generalize well across languages and domains. The paper's findings advocate for continued exploration of hybrid models that incorporate successful elements from prior approaches, such as affix and morphological features.

Looking ahead, future research could focus on further refining neural architectures to handle multilingual and multi-domain data more effectively. Additionally, exploring self-supervised and transfer learning techniques might offer new opportunities to improve NER performance across resource-limited languages.

In conclusion, this comprehensive survey not only delineates the evolution of NER systems but also provides a roadmap for future innovations in leveraging neural networks to tackle complex linguistic tasks efficiently.