RoTRAG: Rule of Thumb Reasoning for Conversation Harm Detection with Retrieval-Augmented Generation

Abstract: Detecting harmful content in multi turn dialogue requires reasoning over the full conversational context rather than isolated utterances. However, most existing methods rely mainly on models internal parametric knowledge, without explicit grounding in external normative principles. This often leads to inconsistent judgments in socially nuanced contexts, limited interpretability, and redundant reasoning across turns. To address this, we propose RoTRAG, a retrieval augmented framework that incorporates concise human written moral norms, called Rules of Thumb (RoTs), into LLM based harm assessment. For each turn, RoTRAG retrieves relevant RoTs from an external corpus and uses them as explicit normative evidence for turn level reasoning and final severity classification. To improve efficiency, we further introduce a lightweight binary routing classifier that decides whether a new turn requires retrieval grounded reasoning or can reuse existing context. Experiments on ProsocialDialog and Safety Reasoning Multi Turn Dialogue show that RoTRAG consistently improves both harm classification and severity estimation over competitive baselines, with an average relative gain of around 40% in F1 across benchmark datasets and an average relative reduction of 8.4% in distributional error, while reducing redundant computation without sacrificing performance.

• The paper introduces RoTRAG, a framework that grounds conversation harm detection using external human-written rules of thumb.
• It employs a three-stage pipeline with dynamic turn-level routing and retrieval augmentation from a large corpus to improve harm assessment.
• Experiments demonstrate significant gains in F1 scores and efficiency, ensuring transparency and consistency in dialogue safety judgments.

RoTRAG: Rule of Thumb Reasoning for Conversation Harm Detection with Retrieval-Augmented Generation

Motivation and Background

Traditional approaches to harm detection in multi-turn dialogue systems generally rely on the internal parametric knowledge of LLMs, sometimes supplemented with explicit reasoning protocols such as Chain-of-Thought (CoT) or collaborative multi-agent frameworks. Despite incremental improvements, these systems are limited by their opacity, inconsistency in socially nuanced contexts, and lack of grounding in external, interpretable normative evidence. Harm judgments in conversation require aggregating intent, escalation, interpersonal cues, and context over several turns, which surpasses the capacity of single-pass or purely parametric methods to yield reliable, explainable, and computationally efficient predictions.

RoTRAG addresses these deficiencies by integrating concise human-written moral norms—Rules of Thumb (RoTs)—retrieved from a structured external corpus to explicitly ground each turn-level harm judgment. This retrieval-augmented design is paired with a lightweight classifier that dynamically routes the reasoning process, invoking RoT retrieval and generation only when warranted by substantive contextual change.

Methodological Framework

RoTRAG is architected as a modular pipeline comprising three stages: (1) turn-level classification and routing, (2) retrieval-augmented RoT generation, and (3) label prediction from accumulated RoT history.

Turn-Level Routing: A fine-tuned, encoder-based classifier (RoBERTa-large) determines whether a dialogue turn necessitates fresh RoT reasoning or can inherit the prior turn’s RoT. The classifier is supervised using high-quality human annotations expanded via an LLM-based prompt refined for high agreement with manual labeling.

Retrieval-Augmented RoT Generation: If new reasoning is triggered, the current turn is summarized with an LLM to obtain an action-oriented query, which is embedded and used to retrieve the top-k semantically similar action–RoT examples from a large corpus (D-Rules-of-Thumb, ~578k items). The RoT generator module then conditions on the current context and retrieved examples to produce a concise, contextually appropriate RoT.

Label Prediction: The history of RoTs, together with the dialogue context up to the current turn, is fed into a classifier or generative prediction module to determine the final harm severity label. This enables decisions to be transparently traceable to explicit external norms.

Empirical Evaluation

Experiments utilize both API and open-source backbones (GPT-4o mini, Qwen3-14B) on ProsocialDialog and Safety Reasoning Multi-Turn Dialogue, representing both classification and severity regression targets.

RoTRAG consistently outperforms zero-shot, CoT, role-based, and advanced multi-agent baselines (Self-Consistency, JailJudge, RADAR) as well as advanced thinking models (GPT-5.4-thinking, Claude 3.7 Sonnet (thinking), DeepSeek-V3.2 (thinking)), which demonstrates that parametric internal reasoning, even at the state-of-the-art, is insufficient for robust multi-turn harm assessment.

Key quantitative results:

• F1 Improvements: Relative gains of 9.6%–81.9% (GPT-4o mini) and 7.6%–56.6% (Qwen3-14B) over the strongest baseline per dataset/backbone.
• Distributional Metrics: Mean Absolute Error, Total Variation Distance, and Earth Mover's Distance are minimized by RoTRAG in 11/12 benchmark settings, establishing superior closeness to gold labels and better preservation of real-world label distributions.
• Efficiency: Routing reduces redundant computation, with meaningful latency and token cost reductions compared to always-on multi-agent methods.

Extensive ablation studies confirm that each component—retrieval augmentation and learned routing—contributes to the system's robustness and efficiency. Full RoT generation for every turn and random routing both degrade performance, highlighting the necessity and sufficiency of selective, norm-grounded reasoning. Direct (parametric-only) RoT generation is helpful but inferior to retrieval-augmented RoT generation for contextual alignment and prediction fidelity.

Qualitative and Analytical Insights

RoTRAG’s intermediate RoT reasoning surpasses CoT in human expert preference (74% in direct comparison), and the quality of generated RoTs is rated highly (mean 4.1/5). Qualitative error analyses of the routing classifier demonstrate that most remaining misclassifications are boundary or ambiguity-driven.

Case studies underscore RoTRAG’s ability to adjust normative focus dynamically as conversation semantics shift, e.g., transitioning from public nudity to misconduct involving minors, which is critical for high-stakes safety domains.

Implications and Theoretical Significance

RoTRAG demonstrates that retrieval-grounded normative reasoning is crucial for achieving context-sensitive, interpretable, and reliable harm assessment in multi-turn conversational AI. The framework stands in contrast to both direct parametric and classical reasoning methods by offering explicit traceability to human social norms:

• Transparency: Each harm judgment is justified by explicit, contextually-relevant external norms.
• Consistency: Retrieval-based grounding reduces prediction instability and digressions in socially nuanced or adversarial contexts.
• Efficiency: Dynamic routing circumvents redundant computation, yielding favorable token-efficiency and inference time profiles suitable for deployment.

Theoretically, this approach positions dialogical harm detection as a hybrid problem—requiring both soft alignment with shared social knowledge (which retrieval provides) and agile, context-dependent reasoning (which routing enables). RoTRAG’s design principles generalize to other tasks requiring the combination of deliberative external evidence and selective inference.

Future research directions include improving the coverage and cultural diversity of RoT corpora, enhancing the specificity of routing beyond binary gating, and integrating richer forms of structural and interpersonal context to further reduce residual misclassification and expand applicability to more complex domains such as negotiation, mediation, or regulatory compliance.

Conclusion

RoTRAG establishes a normative, retrieval-augmented paradigm for harm detection in conversational AI, consistently outperforming internal reasoning and multi-agent prompting strategies by grounding judgments in concise, externally retrieved human norms and dynamically invoking reasoning when conversational context demands. This work demonstrates the necessity of explicit normative evidence for interpretable, consistent safety assessment, and suggests that future AI safety systems should blend retrieval, reasoning, and dynamic control for robust dialogical harm mitigation.