CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systems

Abstract: As LLMs are increasingly deployed as task-oriented agents in enterprise environments, ensuring their strict adherence to complex, domain-specific operational guidelines is critical. While utilizing an LLM-as-a-Judge is a promising solution for scalable evaluation, the reliability of these judges in detecting specific policy violations remains largely unexplored. This gap is primarily due to the lack of a systematic data generation method, which has been hindered by the extensive cost of fine-grained human annotation and the difficulty of synthesizing realistic agent violations. In this paper, we introduce CompliBench, a novel benchmark designed to evaluate the ability of LLM judges to detect and localize guideline violations in multi-turn dialogues. To overcome data scarcity, we develop a scalable, automated data generation pipeline that simulates user-agent interactions. Our controllable flaw injection process automatically yields precise ground-truth labels for the violated guideline and the exact conversation turn, while an adversarial search method ensures these introduced perturbations are highly challenging. Our comprehensive evaluation reveals that current state-of-the-art proprietary LLMs struggle significantly with this task. In addition, we demonstrate that a small-scale judge model fine-tuned on our synthesized data outperforms leading LLMs and generalizes well to unseen business domains, highlighting our pipeline as an effective foundation for training robust generative reward models.

• The paper introduces a synthetic benchmark that rigorously evaluates LLM judges' ability to detect compliance violations in multi-turn dialogues.
• It leverages guideline expansion, adversarial synthesis, and automated dialogue labeling to simulate realistic enterprise conversations with precise ground truth.
• Empirical results reveal domain-specific performance gaps and underscore the importance of fine-tuning for improved context tracking and guideline localization.

CompliBench: Systematic Evaluation of LLM Judges for Compliance Violation Detection in Multi-Turn Dialogue

Motivation and Problem Statement

As LLM-powered agents are increasingly integrated into task-oriented enterprise settings (e.g., customer contact centers), enforcing adherence to nuanced, domain-specific operational guidelines is critical. Compared to generic conversational agents, these real-world systems face stringent requirements—ranging from workflow compliance and professional tone to complex conditional branches driven by regulatory or business constraints. The "LLM-as-a-Judge" paradigm has emerged as a scalable approach for evaluating whether such agents follow operational guidelines. However, the reliability of LLM judges themselves in this high-stakes, multi-turn, fine-grained setting remains largely unassessed due to two outstanding challenges:

1. Absence of Systematic Benchmarks: Without turn-level ground truth for which guideline applies and at which turn violations occur, reliable benchmarking is impossible.
2. Data Scarcity: Human annotation is infeasible at scale, and synthesizing realistic agent violations via LLMs is non-trivial.

CompliBench directly addresses these gaps by proposing a synthetic benchmark and pipeline explicitly designed to probe the detection and localization capacity of LLM judges for compliance violations in complex dialogues.

Figure 1: Illustration of the evaluation framework for LLM-as-Judge in CompliBench.

Methodology: Scalable Guideline-Driven Data Synthesis

To ensure realism and extensibility, CompliBench begins with authentic corporate guidelines, then systematically constructs diverse, high-fidelity dialogic data with controllable violations and precise ground truth.

Guideline Expansion and Deduplication

Seed guidelines are qualitatively expanded via LLMs to cover a broad intent and workflow space. A two-pronged quality control and deduplication strategy ensures that (1) generated guidelines are non-overlapping/non-conflicting and (2) workflow similarity is below a domain-specific threshold, with problematic generations iteratively refined via multi-model LLM feedback. This guarantees structural diversity and precludes semantic collapse within benchmark domains.

Figure 2: Overview of Data Synthesis. Pipeline for scaling, modifying, and applying contact center guidelines to generate high-quality, guideline-driven conversations with automatic labeling.

Violation Injection via Adversarial Synthesis

Agent violations are explicitly simulated by injecting realistic "wrong" guidelines (violation variants) at randomly selected dialogue points. Unlike naively prompted output corruption, CompliBench employs a two-level LLM judge pipeline to optimize for both semantic plausibility and adversarial difficulty:

• Content Consistency Judge: Ensures that behavioral changes in the dialogue are meaningful and textually observable, not merely stylistic.
• Adversarial Compliance Judge: Validates that violations are challenging to detect (i.e., state-of-the-art judges often fail to catch them), thereby resisting trivial pattern recognition.

This iterative loop is continued until difficult, high-quality adversarial violations are synthesized and validated via multi-judge majority voting.

Automated Dialogue Generation and Labeling

Simulated user and assistant agents interact under the modified guidelines, with dialogue acts and agent responses driven by sampled user personas and intents, yielding diverse discourse structures. Each agent turn is automatically labeled with its governing guideline and whether it violates compliance, allowing granular scoring.

Evaluation Metrics and Task Definition

CompliBench defines three rigorous evaluation metrics:

• Strict Guideline Accuracy (SGA): For compliant turns, the LLM judge must correctly identify both the applicable guideline and that no violation occurred.
• Violation Detection Accuracy (VDA): For violated turns, the model must correctly flag a compliance breach (regardless of guideline identification).
• Conversation-Level Accuracy (CLA): The model must get both the governing guideline and violation status correct for every turn in the conversation—i.e., strict end-to-end correctness.

These metrics jointly probe not only violation detection sensitivity but also the judge’s ability to track fine-grained workflow state and context.

Empirical Results: LLM Judges and Generalization

CompliBench evaluates mainline LLM judges (frontier, open-source, and reward models) using synthesized multi-turn conversations across three operational domains: Healthcare, Insurance, and Airline.

Figure 3: Main results of general-purpose LLM judges and our judges across the Healthcare, Insurance, and Airline domains. The size of the bubbles represents the base model scale.

Baselines: State-of-the-Art LLMs and Reward Models

• Frontier LLMs: Gemini-3-pro and GPT-5 achieve the strongest overall performance, particularly on SGA, but show a consistent gap on VDA and even more pronounced degradation on CLA (e.g., top CLA under 60% in best domains).
• Open-source LLMs: Qwen3, GLM-5, and similar models lag proprietary solutions, sometimes by 30+ absolute points.
• Reward Models: Existing classifier and generative reward models, mostly trained on single-turn preference alignment data (e.g., RM-Bench (Liu et al., 2024), RewardBench (Lambert et al., 2024)), perform poorly on multi-turn guideline adherence, with the best models peaking at around 60% SGA and abysmal CLA.

Task-Specialized Fine-Tuning

A compact Qwen3-8B judge fine-tuned with CompliBench-synthesized reasoning traces on 1.4k airline dialogues outperforms generalist frontier models in both CLA and VDA, achieving robust generalization to out-of-domain (Healthcare, Insurance) test corpora. Notably, domain transfer is strong, highlighting the pipeline’s value for reward-model pretraining.

Systematic Error Analysis

Manual categorization of judge errors exposes persistent, non-trivial bottlenecks in current LLM evaluation:

• Scope Mis-Attribution: >50% of errors result from the judge misidentifying the correct guideline/phase—especially in workflows with subtle conditional similarity.
• Overly Strict vs. Lax Judging: Judges are systematically over-strict on compliant turns (e.g., hallucinating missing requirements, enforcing overly literal guideline readings) but overly lax in violation detection (accepting non-equivalent behaviors as compliant).
• Evidence Integration and Chain-of-Reasoning Errors: Errors are often due to the model lacking robust long-context state tracking, leading to failures in integrating evidence across multiple turns.

Appendix results substantiate the above categories with turn-annotated dialogue excerpts and cross-domain breakdowns.

Implications and Future Directions

CompliBench reveals that even SOTA LLMs have severe limitations as compliance judges in real-world, multi-turn agent evaluation—especially regarding guideline localization, context integration, and violation detection tradeoffs. This calls into question the reliability of deploying LLM-judged compliance monitors in regulatory or business-critical settings without additional controls.

Practically, synthesized data pipelines akin to CompliBench provide a foundation for architecting more robust, transparent reward models—especially when fine-tuning with adversarially challenging, structurally diverse supervision, rather than relying solely on natural dialogues or subjective preference data. The adversarial pipeline for flaw injection and guideline augmentation is extensible to new domains and compliance regimes.

Theoretically, further advances are needed in LLM architectures and training objectives to represent and reason about stateful, conditional execution traces and to robustly disambiguate overlapping guideline semantics. Direct modeling of long-range conversational dependencies and explicit workflow state tracking may be required.

Conclusion

CompliBench provides a rigorous, extensible benchmark for evaluating LLM judges in compliance violation detection across multi-turn enterprise dialogue. Through scalable synthesis of guideline-driven, adversarially optimized, and fully labeled dialogues, it exposes the limitations of both generalist and reward-model LLMs—while also demonstrating that targeted fine-tuning yields substantial gains. The principal challenge remains correctly localizing the applicable guideline phase and integrating long-context evidence, with existing LLMs showing systematic judgment asymmetries. CompliBench facilitates both the development and assessment of future models intended for high-stakes, guideline-governed dialogue evaluation.

(2604.12312)