- The paper presents a personalised moderation framework (PRISM) that leverages multi-agent inference to model individual sensitivity profiles.
- It employs adaptive, user-in-the-loop learning with exponential moving averages to dynamically calibrate thresholds for sentiment, violence, and hate content.
- Experimental results show significant improvements, achieving 81.5% macro F1 and 99.73% hate recall, challenging traditional universal moderation methods.
Multi-Agent Personalised Inference for Content Moderation: Algorithmic and Policy Implications
Motivation and Background
The exponential growth of digital platforms has intensified challenges in governing harmful content and safeguarding digital well-being. Conventional moderation architectures employ centralised, monolithic classifiers with fixed thresholds, failing to capture the subjective and context-dependent nature of harm perception. This paper proposes a perspectivist, user-centred moderation paradigm, introducing personalised inference by modelling explicit sensitivity profiles per user. The approach is situated within a broader lineage of hybrid moderation research, responding to the epistemological complexities of ground truth, the ethical trade-offs of platform governance, and the operational demands of scalable user agency.
System Architecture and Mechanisms
The PRISM framework operationalises a user-in-the-loop methodology to generate and continuously adapt granular sensitivity profiles for each individual. The architecture features:
- User Profile Construction: Profiles consist of dimension-specific thresholds, importance weights, and confidence scores across domains such as sentiment, insult, violence, and dehumanisation. Adaptive learning dynamically calibrates these thresholds via explicit user feedback, utilising an exponential moving average for convergence.
- Multi-Agent Inference Engine: Content is routed through a Manager Agent which orchestrates domain-specific Expert Agents (Sociologist, Linguist, Psychologist), each receiving a composite prompt contextualised by user parameters. This enables dimensionally decomposed analysis unavailable to monolithic classifiers. A Ghost Profile Agent is invoked for tie-breaking and perspective simulation, ensuring alignment with user's subjective thresholds. The Synthesis Agent aggregates predictions for final binary classification.
- Continuous Online Learning: Feedback-driven threshold modification ensures rapid adaptation, with confidence weighting for new users and population priors mitigating cold-start effects.
- Scalable Design: Profiles and their updates are managed in a SQLite database, permitting integration with real-world platforms and low-latency deployment.
Experimental Evaluation
The framework was empirically validated using the Measuring Hate Speech dataset, which leverages fine-grained multidimensional annotation and preserves individual annotator judgments. Key methodological features include:
- Stratified User Sampling: 100 profiles were constructed to span annotation volume, severity, and demographic diversity, mitigating statistical unreliability due to low annotation counts.
- Perspectivist Ground Truth: Each user's annotation is treated as valid, abandoning universal label paradigms in favour of epistemological pluralism.
- Comparative Baselines: Experiments compared PRISM against universal filters (population mean), single-agent personalised inference, and a suite of state-of-the-art transformer-based models (BERTweet, TimeLMs, RoBERTa, BERT) [Sachdeva et al., 2022].
Quantitative Results
Two bold numerical outcomes define the system's contribution:
- Accuracy Gains: PRISM achieves a 81.5% macro-averaged F1, a 31.9% relative improvement over universal baseline (61.8%), and outperforms all tested transformer baselines (BERTweet at 80.5%) (2605.01416). The multi-agent design yields disproportionate gains in precision (85.2%) compared to recall, highlighting dimensional reasoning benefits.
- Recall for Hate Speech: The system attains 99.73% recall for hate content, effectively shielding end-users from content exceeding their sensitivity thresholds at the cost of moderate over-filtering (70.69% precision).
Learning curve analysis shows immediate performance advantages for personalised baselines even at minimal feedback (k = 2), mitigating cold-start issues, with further gains as dimensionality increases through multi-agent deliberation.
Theoretical and Practical Implications
This work fundamentally repositions the locus of moderation, from platform-enforced universalism to user-centred perspectivism. Algorithmically, it demonstrates that explicit, dynamic user sensitivity modelling is superior to universal policies or static personalisation strategies. Multi-agent architectures increase interpretability and allow domain-specific reasoning, facilitating transparency and contestability in automated moderation.
Practically, the system entails complex policy trade-offs. While user agency is enhanced, there is a risk in shifting the burden of moderation onto individuals, especially vulnerable groups. The authors propose 'opt-out' mechanisms for severe content—retaining baseline platform protections—and enabling profile subscriptions from trusted civil society groups, balancing autonomy with cognitive load mitigation.
Adoption of this paradigm could substantially improve digital well-being, reduce representational bias, and support more equitable platform governance. However, scaling such systems requires careful design to avoid exacerbating trauma, ensure fairness, and accommodate evolving social norms.
Future Directions
Integration with mainstream platforms may foster hybrid governance architectures in content moderation, combining population-level safety standards with user-driven adaptability. There is scope for expanding the feedback loop to include more implicit signals, crowdsourcing community-based profiles, and enhancing precision of sensitivity modelling. Improvements in theory-of-mind capabilities in LLMs and richer context engineering could further amplify performance and enable dynamic group-based moderation. Long-term, these developments may underpin more democratic, pluralistic structures for digital governance and algorithmic policy.
Conclusion
By reconceptualising harmfulness as a pluralistic, user-dependent property and embedding explicit individual agency into a multi-agent inference pipeline, this research advances algorithmic moderation both quantitatively and qualitatively. The PRISM framework demonstrates substantial improvements in alignment with user sensitivities and overall moderation efficacy compared to universal and static baselines. Theoretical implications challenge foundational assumptions in content moderation and pave the way for more transparent, fair, and adaptive platform governance. The multi-agent, personalised paradigm is likely to influence future design of moderation architectures in both academic and industrial contexts.