Detecting Online Hate Speech Using Context Aware Models (1710.07395v2)

Abstract: In the wake of a polarizing election, the cyber world is laden with hate speech. Context accompanying a hate speech text is useful for identifying hate speech, which however has been largely overlooked in existing datasets and hate speech detection models. In this paper, we provide an annotated corpus of hate speech with context information well kept. Then we propose two types of hate speech detection models that incorporate context information, a logistic regression model with context features and a neural network model with learning components for context. Our evaluation shows that both models outperform a strong baseline by around 3% to 4% in F1 score and combining these two models further improve the performance by another 7% in F1 score.

• The paper demonstrates that integrating contextual cues into both logistic regression and neural networks enhances hate speech detection, with ensemble models reaching a 10% F1-score improvement.
• The methodology uses LSTM networks with attention and enriched feature sets like LIWC and emotion lexicons to capture the nuanced context of user comments.
• The study’s results highlight the practical value of context-aware models for developing more effective online hate speech moderation systems.

Context-Aware Models for Detecting Online Hate Speech: An Analysis

As the prevalence of hate speech continues to increase in online spaces, the need for effective automatic detection methods is critical. The paper "Detecting Online Hate Speech Using Context Aware Models" addresses the challenge of identifying hate speech by incorporating contextual information into detection models. The paper presents innovative approaches using both logistic regression and neural network models, achieving a significant improvement in detecting hate speech over traditional methods.

Dataset and Methodology

The authors introduce a novel dataset, the Fox News User Comments corpus, which comprises 1,528 user comments from 10 widely discussed Fox News articles. This dataset is distinct in its inclusion of rich contextual information for each comment, such as the user's screen name, the comment thread, and the corresponding news article. This contextual data is pivotal in understanding the subtleties and often implicit nature of hate speech online.

Two primary types of models are explored in the paper: context-aware logistic regression models and neural network models. The logistic regression models incorporate context by utilizing features extracted from both the target comment and its context (usernames and article titles). Enhanced feature sets include word-level and character-level n-grams, LIWC, and NRC emotion lexicons. Conversely, the neural network models leverage Long Short-Term Memory (LSTM) networks with attention mechanisms to capture the compositional meaning of comments and context.

Experimental Results

The evaluation, conducted via 10-fold cross-validation, reveals that incorporating context results in notable performance improvements. Both logistic regression and neural network models outperform the baseline models by approximately 3-4% in F1 score. Remarkably, the ensemble of both models further enhances performance, achieving about a 10% improvement over the baseline in F1-score.

1. Logistic Regression Models: Contextual features extracted from usernames and titles significantly improved F1 scores, demonstrating the utility of contextual data in hate speech detection.
2. Neural Network Models: The introduction of context through LSTMs with attention mechanisms proved particularly effective, with the best model incorporating news title context yielding superior results.
3. Ensemble Models: Combining logistic regression and neural network models capitalized on the unique strengths of each approach, resulting in optimal hate speech detection performance.

Implications and Future Directions

This research underscores the vital role of context in detecting the nuanced and implicit nature of online hate speech. By highlighting the limitations of models that solely rely on textual features without contextual understanding, the authors advocate for the development of more sophisticated, context-aware models.

The implications of this paper are both practical and theoretical. Practically, improved hate speech detection models can aid online platforms in moderating content more effectively, fostering safer online environments. Theoretically, this work contributes to the broader understanding of the linguistic and contextual features that characterize hate speech, paving the way for future research in natural language processing and machine learning.

Future work could extend this approach by exploring more diverse datasets, employing advanced neural architectures, or integrating multimodal data for even richer context understanding. As the fight against online hate continues, the incorporation of contextual information into detection systems holds significant promise for advancing the field of automatic hate speech detection.