Fine-Grained Analysis of Propaganda in News Articles (1910.02517v1)

Abstract: Propaganda aims at influencing people's mindset with the purpose of advancing a specific agenda. Previous work has addressed propaganda detection at the document level, typically labelling all articles from a propagandistic news outlet as propaganda. Such noisy gold labels inevitably affect the quality of any learning system trained on them. A further issue with most existing systems is the lack of explainability. To overcome these limitations, we propose a novel task: performing fine-grained analysis of texts by detecting all fragments that contain propaganda techniques as well as their type. In particular, we create a corpus of news articles manually annotated at the fragment level with eighteen propaganda techniques and we propose a suitable evaluation measure. We further design a novel multi-granularity neural network, and we show that it outperforms several strong BERT-based baselines.

• The paper introduces a fragment-level detection methodology using expert annotations and a taxonomy of eighteen propaganda techniques to improve news analysis.
• The study developed a comprehensive corpus and annotation scheme that enables more nuanced detection than traditional whole-article approaches.
• Experimental results show that a novel multi-granularity neural network outperforms BERT-based methods by effectively combining sentence- and token-level insights.

Fine-Grained Analysis of Propaganda in News Articles

The paper "Fine-Grained Analysis of Propaganda in News Articles" introduces an advanced approach for detecting propaganda at a more detailed level than previous research efforts, which typically focused on classifying entire articles as propagandistic or not based solely on their source. This work highlights the limitations of that methodology, particularly noting the lack of granularity and potential mislabeling of content due to the generalization of source biases.

Key Contributions

1. Corpus Creation: The authors developed a corpus of news articles annotated at the fragment level with eighteen distinct propaganda techniques. This data collection involved meticulous manual annotations by experts to ensure high quality and reliability.
2. Taxonomy of Propaganda Techniques: The paper provides a comprehensive taxonomy of propaganda techniques. By targeting specific propagandistic techniques rather than entire articles or sources, this paper offers a more precise tool for analysis.
3. Neural Network Architecture: The paper proposes a novel multi-granularity neural network designed to outperform BERT-based baselines in identifying propaganda instances. This architecture effectively leverages hierarchical information, combining both sentence-level and token-level insights to yield better performance.
4. Evaluation Metrics: New evaluation metrics have been developed to accommodate the nuances of the task, especially considering partial overlaps of annotations. Traditional precision and recall metrics are adapted to handle the variable length of identified spans and overlapping techniques.

Methodology

The paper recognized inherent issues in prior methodologies that labeled entire outlets as propagandistic. This often resulted in misleading conclusions since outlets might produce both biased and unbiased content. The authors counter this by employing expert annotations for specific text fragments with predefined propaganda techniques.

A sophisticated annotation scheme and guidelines ensured consistency across annotations, with the paper involving a two-stage annotation process for reliability.

Experimental Results

In experimental evaluations, the proposed multi-granularity network demonstrated superior performance relative to BERT-only models. It showed improved precision in detecting propaganda techniques by efficiently combining signals from different text granularities.

The multi-granularity model's design allows it to selectively use sentence-level predictions to enhance token-level propaganda detection. The results indicate that this model strikes a balance in utilizing hierarchical information effectively.

Implications and Future Directions

The implications of this work are substantial, providing a foundation for the development of more nuanced, explainable AI systems that could significantly enhance the transparency of news content analysis. By allowing users to see exactly which segments of text are associated with propagandistic techniques, such systems could improve media literacy.

Future research may focus on expanding the corpus and applying similar techniques beyond the scope of news articles to other media forms. Additionally, as AI models evolve, incorporating transformers or other state-of-the-art models optimized for linguistic inference could refine and expand these initial findings.

The work invites collaborative research into the intersection of propaganda detection, argumentation mining, and sentiment analysis, potentially fostering a multidisciplinary approach to enhancing media integrity.

In summary, this paper sets a considerable advancement in propaganda analysis by proposing a detailed, structured methodology that can discern the multifaceted ways that propaganda manifests in media texts.