MADQA: Multi-Agent Debate & Anomaly Detection

• MADQA is a multifaceted term in AI, encompassing quantum measurement anomaly detection, document QA benchmarking, and multi-agent debate systems for diverse reasoning tasks.
• The document QA benchmark component features 2,250 human-authored Q&A pairs over heterogeneous PDF documents, emphasizing multi-hop reasoning and rigorous validation.
• The multi-agent debate approach in MADQA utilizes reinforcement learning for dynamic topology control, achieving efficiency gains and robust transfer across diverse QA tasks.

MADQA is a term with multiple technical meanings across machine learning and artificial intelligence, including (1) a quantum-measurement inspired algorithm for streaming anomaly detection; (2) a benchmark and protocol for evaluating multimodal agents and humans in document-intensive question answering; and (3) a class of multi-agent debate systems for question answering with topology optimization via reinforcement learning. Each instantiation is associated with a distinct methodological and empirical tradition.

1. Quantum Measurement Anomaly Detection (Incremental QMAD/MADQA)

The Incremental Quantum Measurement Anomaly Detection (InQMAD, often also labeled as MADQA in terminology) algorithm is a streaming anomaly detection method that implements a continuous kernel density estimator via random Fourier features and quantum measurement operations. Using a fixed-dimensional random Fourier embedding $\bm\phi(\mathbf x)$ of the input and maintaining a feature-space density matrix $\rho_t$ updated by exponential moving average,

$$\rho_{t+1} = (1-\alpha)\,\rho_t + \alpha\,\bm\phi(\mathbf x_{t+1})\,\bm\phi(\mathbf x_{t+1})^T,$$

the anomaly score for a candidate $\mathbf x$ is

$$s_t(\mathbf x) = \bm\phi(\mathbf x)^T \rho_t \bm\phi(\mathbf x),$$

interpreted as the quantum measurement probability of state $\bm\phi(\mathbf x)$ against the empirical density. Low $s_t(\mathbf x)$ indicates anomalous samples.

This approach achieves $O(1)$ update complexity per streaming sample by operating in a fixed $D$-dimensional feature space, and is adaptive both to concept drift and to subspace expansion via adaptive refinement of Fourier features. Empirical evaluation on twelve datasets demonstrates state-of-the-art AUC-ROC, outperforming alternatives such as MemStream and RCF, especially under high-frequency outliers and concept drift (Gallego-Mejia et al., 2022).

2. MADQA as a Benchmark for Agentic Question Answering

In document reasoning, MADQA refers to a benchmark and associated evaluation protocol that measures the strategic capabilities of multimodal agents (and humans) in answering questions grounded in diverse, heterogeneous document collections. The MADQA dataset comprises 2,250 carefully constructed, human-authored question–answer pairs over a corpus of 800 PDF documents diverse in domain, layout, and scale (median 5, mean 23.3 pages; total 18,619 pages) (Borchmann et al., 12 Mar 2026).

Benchmark Construction

• Document corpus: PDF documents, 63 fine-grained types in 13 top-level domains selected from DocumentCloud; layouts span table-dense financial/government records, figure-heavy technical files, and text-dense legal sources.
• Question annotation: Human labelers spent 1200+ hours authoring Q&A pairs, tagging minimal page-level evidence for each. Questions are non-trivial and unambiguous; ~17% are multi-hop (either cross-page or cross-document).
• Validation: Oracle-based LLM and expert reviews ensure accuracy; lexical overlaps confirm that n-gram matching and answer guessing offer limited utility.

Classical Test Theory-driven Split

Difficulty ($p_j$) and point-biserial discrimination were used to select items, reserving the hardest questions ("sentinel pool") for test set headroom. The final split provides 1,550 train, 200 dev, and 500 test questions, with test achieving high rank correlation ($\rho_t$0) to the full dataset and 100 unsolved sentinel items.

3. Evaluation Protocol and Metrics for Agentic Search

The MADQA evaluation protocol operationalizes both extractive accuracy and agentic behavior:

• Accuracy: Two-stage metric (exact matching, then semantic LLM judge) with calibrated human-LM agreement ($\rho_t$1) and Rogan-Gladen bias correction.
• Grounding Attribution: Page F1 and Doc F1 metrics measure overlap between agent-cited and oracle-labeled evidence; divergence diagnoses fine-grained navigation failure.
• Effort Calibration: The number of discrete search/operator steps per question is tracked per agent. Calibration is quantified by the Kuiper statistic

$\rho_t$2

where $\rho_t$3 is the deviation from mean accuracy as effort increases. Low $\rho_t$4 indicates efficient, stable effort alignment.

4. Experimental Findings and Strategic Gaps

On the MADQA leaderboard, best-in-class agentic systems (Gemini 3 Pro BM25 Agent, 82.2% accuracy, Kuiper 25.8) match aggregate human performance (82.2%, Kuiper 14.6) but show low item agreement ($\rho_t$5) and a consistent 17–20% gap to oracle retriever + human reasoning (99.4%). Humans demonstrate higher “cold start” efficiency—attaining 50% accuracy on the first query, while agents lag behind (12%).

Error decomposition attributes failures to:

• Retrieval failures (35.7%): inability to surface relevant content.
• Comprehension errors (28.8%): misreading or misinterpreting retrieved evidence.
• Navigation failures (23.0%): last-mile traversal problems, particularly in dense or cross-document layouts.
• Refusals (12.6%): abstaining from answering.

Agents frequently enter unproductive query reformulation loops (high Kuiper), in contrast to humans' strategic abandonment or redirection. Visual grounding is a further limiting factor, with 58% of queries requiring artifact/layout understanding unmet by text-only models.

5. Multi-Agent Debate (MADQA) and Topology Optimization

MADQA is also associated with a class of Multi-Agent Debate Question Answering systems, particularly in the context of dynamic communication topology control for collaborative reasoning (Wang et al., 27 Feb 2026). The RUMAD framework formulates the multi-agent debate process as a Markov Decision Process, with a PPO-trained controller adaptively selecting the communication graph at every debate round based on content-agnostic observations (reasoning embeddings, answer choices).

The RUMAD approach employs a multi-objective reward function that blends solution accuracy, consensus, efficiency, and communication sparsity. By enforcing dual thresholds on agent activation (self-confidence vs. group influence) and structuring agent prompts by semantic-magnitude tiers (Critical, Reference, Background), RUMAD attains substantial token cost reductions (e.g., 66.2% fewer tokens at $\rho_t$6 while improving accuracy to 32% on GPQA) and exhibits robust zero-shot transfer to new QA tasks (MMLU, GSM8K, GPQA).

Principles emerging from this approach include the importance of content-agnostic control for neutrality, explicit reward balancing for multi-objectivity, soft budget regularization for generalizability, and the effectiveness of dual-threshold deactivation for communication efficiency.

6. Broader Implications and Future Directions

MADQA, as instantiated in both document-question answering and multi-agent debate, exposes structural bottlenecks in modern reasoning systems:

• Oracle gap persistence: Even the most capable agents are retrieval-bound, with state-of-the-art systems plateauing at ~80–82%—the retrieval pipeline, especially for cross-page/cross-doc and visually grounded items, is central.
• Calibration and planning: Current agents exhibit high variance in effort calibration; robust exploration/exploitation balances (e.g., via RL with tool feedback and episodic memory modules) are required to improve alignment.
• Hybrid and multimodal retrieval: Integrating sparse, dense, and late-interaction retrieval, along with layout-sensitive encoders, is critical for bridging the performance gap on structured/visual evidence questions.
• Benchmark evolution: Sustained progress requires ongoing hardening of evaluation datasets (sentinel pool expansion, adversarial and multilingual inclusion) and the development of truly agentic exploration/abandonment policies.

A plausible implication is that hybrid human–agent pipelines, or agent ensembles structured by reinforcement-learned communication policies, can outperform either component alone, provided that retrieval and comprehension errors are partially disjoint and ensemble agreement is optimized. The empirical findings from MADQA benchmarks and multi-agent debate experiments directly inform the design of future agentic architectures and evaluation protocols for document reasoning tasks (Gallego-Mejia et al., 2022, Borchmann et al., 12 Mar 2026, Wang et al., 27 Feb 2026).