MAD Framework: Judge-Based Adaptive Break

• The paper introduces a mechanism where a discriminative judge applies strict confidence thresholds to adaptively terminate debate rounds, boosting efficiency.
• The framework leverages tit-for-tat argumentation among multiple agents to foster divergent thinking and mitigate degeneration-of-thought issues.
• Empirical analysis shows that adaptive breaks improve solution quality and computational efficiency by halting debates once high-confidence answers emerge.

The Discriminative Judge–Based Adaptive Break is the core mechanism enabling the Multi-Agent Debate (MAD) framework to foster divergent thinking in LLMs and optimally adjudicate complex reasoning tasks. By introducing a discriminative judge component that adaptively terminates debate rounds contingent on the emergence of a sufficiently confident solution, this architecture addresses the Degeneration-of-Thought (DoT) limitation of self-reflective LLM protocols. The adaptive break logic, operationalized through precisely defined confidence thresholds, allows the debate to conclude as soon as an argument is established as correct with high certainty, thereby improving both computational efficiency and decision robustness (Liang et al., 2023).

1. Structural Overview of the MAD Framework

In MAD, $N$ independent debater agents $\{D_1, ..., D_N\}$—each instantiated by the same LLM architecture—alternately respond to a shared task prompt $t$, with each round comprising a tit-for-tat exchange of arguments. Crucially, a single judge agent $J$ (an LLM) observes the debate. The judge alternates between two operational modes:

• Discriminative Mode ($J_d$): After each round of $N$ new arguments, the judge evaluates the debate state to determine if a correct solution has emerged, using a confidence scoring mechanism.
• Extractive Mode ($J_e$): Upon debate termination (via adaptive break or reaching the maximum round cap $M$), the judge extracts and outputs the final answer from the debate record.

Rather than relying on majority voting, $J$ acts as a discriminative judge, searching for a candidate answer that meets stringent confidence criteria before halting the debate (Liang et al., 2023).

2. Formalization of the Discriminative Scoring and Adaptive Break

Let the debate history at round $m$ be $\{D_1, ..., D_N\}$0. For each new argument $\{D_1, ..., D_N\}$1, the judge computes a confidence score: $\{D_1, ..., D_N\}$2 where $\{D_1, ..., D_N\}$3 is realized by prompting the judge LLM with a calibrated query requesting its degree of certainty that $\{D_1, ..., D_N\}$4 solves the task.

The adaptive break is governed by two criteria at every round $\{D_1, ..., D_N\}$5:

• Absolute threshold: $\{D_1, ..., D_N\}$6
• Relative margin threshold: $\{D_1, ..., D_N\}$7

The discriminative judge invokes an adaptive break as soon as either criterion is satisfied: $\{D_1, ..., D_N\}$8 The first $\{D_1, ..., D_N\}$9 for which $t$0 marks the debate stopping point. Otherwise, the process continues until the cap $t$1 is reached (Liang et al., 2023).

3. Detailed Process: Pseudocode and Round Dynamics

The following algorithmic flow governs one complete MAD run: $J$8 Debaters are primed to engage in tit-for-tat argumentation, directly countering each other's preceding claims to maximize constructive disagreement. The judge’s confidence scoring after each round dynamically determines whether additional debate is warranted or a satisfactory answer has been reached (Liang et al., 2023).

4. Control Parameters and Their Impact

The key hyperparameters regulating the discriminative judge and adaptive break are:

Hyperparameter	Typical Range/Setting	Functional Role
$t$2 (agents)	$t$3 (default: 2)	More agents increase viewpoint coverage, but also scoring complexity.
$t$4 (max rounds)	$t$5	Impacts depth of debate; excessive $t$6 can incur wasted computation.
$t$7	$t$8	Ensures high judge confidence before terminating.
$t$9	$J$0	Ensures sufficient margin to declare a leader in close debates.
Tit-for-tat depth	meta-prompted	Moderate disagreement (stance $J$1) yields best results; extremes harm performance.

Modulating these values affects the likelihood, timing, and quality of the adaptive break. Moderately enforced tit-for-tat interaction fosters the emergence of divergent, yet constructive, argumentation, which is crucial for the judge’s effective discrimination (Liang et al., 2023).

5. Empirical Analysis and Ablation Studies

Ablation and controlled studies substantiate the efficacy of the discriminative judge–based adaptive break:

• Iteration-wise performance: For challenging commonsense MT tasks, extracting an answer only after a round reaches the adaptive break point achieves a peak COMET score (e.g., 82.0 at $J$2), outperforming any fixed-round baseline and confirming that overlong debates not only waste budget but also degrade final answer quality.
• Tit-for-tat intensity: Moderate disagreement ($J$3) achieves top human accuracy ($J$4), whereas both consensus ($J$5) and forced polarization ($J$6) reduce performance.
• Judge bias: When judge and debaters are the same LLM, the judge displays systematic preference ($J$7) for the correcting (negative) agent. When the judge differs architecturally from at least one agent, it favors the agent matching its own architecture, suggesting limits in neutrality (Liang et al., 2023).

Empirical results reinforce that adaptive debate stopping, orchestrated by a discriminative judge, is essential for robust, sample-efficient selection of high-quality solutions in multi-agent LLM settings.

6. Theoretical and Practical Significance

The discriminative judge–based adaptive break is the linchpin that enables MAD to:

• Terminate the debate efficiently upon emergence of a high-confidence solution, conserving computational resources.
• Avoid over-debating, which leads to performance degradation or circular argumentation.
• Selectively promote correct, well-substantiated chains-of-thought in the presence of multiple independent proposals.
• Encourage and harness divergent thinking, mitigating the DoT effect endemic to monologic self-reflection in LLMs.

Taken together, these attributes yield state-of-the-art gains for reasoning-intensive benchmarks and illuminate design principles for future multi-agent LLM systems wherein selective, confidence-driven adjudication is essential (Liang et al., 2023).