DRAssist: AI Decision Support in Disputes

• DRAssist is a decision-support system that uses multi-stage LLM pipelines to convert unstructured legal texts into structured dispute summaries.
• It produces three levels of advisory output—identifying the stronger party, demand-wise decisions, and argument strength assessments—to guide adjudicators.
• The system integrates structured summarization with zero-shot multi-level prompting, enhancing transparency and supporting legal reasoning without replacing human judgment.

DRAssist is a prototype decision-support system that uses LLMs to assist human adjudicators in resolving disputes. It is designed around two concrete domains—automobile insurance disputes in India and domain name disputes at WIPO—and converts unstructured case materials into structured dispute summaries before producing three levels of advisory output: identification of the overall stronger party, demand-wise acceptance or rejection decisions, and argument-wise strength assessments. The system is explicitly framed as assistance rather than replacement: its outputs are non-binding, intended for review by human experts, and accompanied by textual justifications (Pawar et al., 2 Sep 2025).

1. Scope, objectives, and conceptual orientation

DRAssist targets disputes in which two parties disagree over facts, responsibilities, or entitlements and a human forum ultimately resolves the case. The paper focuses on automobile insurance disputes between an insurance company and an insured party, and domain name disputes between a complainant and a respondent under the UDRP framework. In both settings, the adjudicator must disentangle agreed facts, identify disagreement points, inspect demands and arguments from both sides, and arrive at a reasoned resolution. DRAssist operationalizes this workflow as a structured, multi-stage LLM pipeline rather than as a single-shot outcome predictor (Pawar et al., 2 Sep 2025).

A central feature of the system is its multi-level notion of resolution assistance. Instead of emitting only a winner label, DRAssist produces three outputs. First, it identifies the overall stronger party. Second, it evaluates each specific demand by each party as ACCEPTED or REJECTED. Third, it evaluates each argument by each party as STRONG or WEAK. This decomposition aligns the model’s outputs with recognizable components of adjudicative reasoning and makes the resulting recommendations more inspectable than conventional judgment-prediction systems.

A common misconception is to treat DRAssist as a direct legal judgment predictor. The system is not presented in that form. It first structures disputes into standardized elements, then prompts LLMs to reason over those elements, and only then derives party-level, demand-level, and argument-level recommendations. This suggests an emphasis on explainable assistance rather than opaque classification.

2. Structured dispute representation

The first major subsystem is structured summarization. Raw case text—court orders for automobile insurance or WIPO decisions for domain names—is converted into a schema that separates agreed facts, disputed aspects, demands, arguments, legal references, and outcomes. For automobile insurance disputes, the schema contains thirteen elements:

• Facts agreed by both parties.
• Aspects on which the parties disagree.
• Demands of the insurance company.
• Demands of the insured party.
• Arguments of the insurance company.
• Arguments of the insured party.
• Relevant prior cases referred, with short summary.
• Relevant statutes or policy terms and conditions referred, with short summary.
• Decision by District Commission.
• Decision by State Commission.
• Final decision by the National Commission with respect to each demand of both parties.
• Justification or rationale for the final decision.
• Winning party.

For domain name disputes, the structure is analogous, but party names are changed to complainant and respondent, prior cases are dropped, and the multi-level commission fields are absent because WIPO decisions are single-level. The resulting summary can be regarded as a record of the form

$$\text{Summary}=\{\text{facts},\text{disagreement\_aspects},\text{demands}_{P_1},\text{demands}_{P_2},\text{arguments}_{P_1},\text{arguments}_{P_2},\ldots,\text{winning\_party},\text{justification}\},$$

with each field represented as natural-language text rather than symbolic logic (Pawar et al., 2 Sep 2025).

The summarization pipeline uses three LLMs—Mistral-7B-Instruct, Meta-Llama-3-8B-Instruct, and GPT-4o-mini—to generate candidate structured summaries from raw dispute documents. A “super summary” is then created element by element: for each field, GPT-4o-mini merges the three candidate outputs into a majority-consistent synthesis. For the winning-party field, the final label is obtained by simple majority vote across the three candidate summaries. This aggregation is intended to reduce omissions and hallucinations relative to any single model.

For approximate summarization evaluation, the paper uses rhetorical-role labels from the opennyai pipeline to derive pseudo-reference segments from the original judgments. In the automobile-insurance corpus, the generated summaries achieve ROUGE-1 F1 $=0.35$, ROUGE-L F1 $=0.22$, and BERTScore F1 $=0.63$ for facts and disagreement aspects, and ROUGE-1 F1 $=0.40$, ROUGE-L F1 $=0.22$, and BERTScore F1 $=0.62$ for arguments. Because the reference texts are not human-authored summaries but rhetorical-role slices, these numbers are best interpreted as approximate structural fidelity rather than definitive summarization quality (Pawar et al., 2 Sep 2025).

3. Prompting strategies and reasoning tasks

Once a dispute has been structured, DRAssist applies zero-shot prompting with three progressively richer strategies. Decision-related fields such as the final decision, winning party, and final justification are removed from the input to avoid leakage of ground truth into the prompts. The LLM must therefore reason from facts, disagreements, arguments, demands, prior cases, and statutes or policy terms alone (Pawar et al., 2 Sep 2025).

The first strategy, $S_1$, is direct prompting for the overall stronger party. It uses agreed facts, disagreement aspects, arguments of both parties, and legal references, and asks the model to identify the stronger party with a short justification. The second strategy, $S_2$, extends this by adding the demands of both parties and requiring demand-wise ACCEPTED/REJECTED labels in addition to the stronger-party decision. The third strategy, $S_3$, adds an explicit chain-of-thought-style decomposition: the model must first classify each argument as STRONG or WEAK, then infer the stronger party, and then decide each demand using the preceding analysis.

The $=0.35$0 prompt defines a STRONG argument as one that is supported by verifiable facts and credible evidence, is logically coherent and internally consistent, and is supported by either some prior case or some statute or policy terms and conditions. It also specifies that a strong argument is legally more sound, provides a clearer interpretation of reality, and is more likely to influence the final decision than a weak argument. Arguments lacking evidence, contradicting facts, or exhibiting weak logic are treated as WEAK. This explicit criterion set makes the prompting strategy more constrained than ordinary free-form legal reasoning.

For demand-wise decisions, the prompts impose a soft consistency requirement: opposing or conflicting demands from opposite parties should not both be accepted. The specified output format is fully structured, with separate sections for each party’s arguments, the stronger-party conclusion, and each party’s demands. This structured prompting makes DRAssist closer to a staged reasoning pipeline than to a single generative answer.

4. Data, labels, and evaluation protocol

The automobile-insurance corpus $=0.35$1 contains 104 English-language disputes from India’s National Consumer Disputes Redressal Commission, filtered to automobile insurance and cases with a clear winner. The domain-name corpus $=0.35$2 contains 351 English-language WIPO disputes from 2021 in which the respondent filed substantive arguments. The two corpora differ substantially in scale and textual length (Pawar et al., 2 Sep 2025).

Corpus	Size	Notes
$=0.35$3	104 disputes	Mean 78.9 sentences, 2343.1 words
$=0.35$4	351 disputes	Mean 121.2 sentences, 3448.8 words

The class distributions are imbalanced. In $=0.35$5, the insurance company wins 35 disputes and the insured party wins 69. In $=0.35$6, the complainant wins 234 disputes and the respondent wins 117. This imbalance motivates the use of Macro-F1 in addition to accuracy. The reported metrics are

$=0.35$7

and

$=0.35$8

Ground-truth labels for stronger party come from the actual case outcomes. Demand-wise and argument-wise gold labels are generated separately by prompting GPT-4o-mini with the full decision text, including the final outcome and reasoning, and asking whether each demand was actually accepted or rejected and whether each argument was effectively endorsed by the tribunal. Because LLM outputs rephrase arguments and demands, matching between predicted and reference items is performed using all-mpnet-base-v2 sentence embeddings and linear sum assignment over cosine-distance cost matrices. This alignment procedure is necessary because simple string equality would be unstable under paraphrase.

Baselines are majority and random predictors for stronger-party, demand, and argument tasks. Justification quality for stronger-party predictions is evaluated only on cases where the stronger-party label is correct, using ROUGE-1 F1, ROUGE-L F1, and BERTScore F1.

5. Empirical performance

The strongest results arise from the most structured prompting strategy. In automobile-insurance disputes, the best stronger-party result is obtained by $=0.35$9 with an ensemble over Mistral-7B-Instruct, GPT-4o-mini, and Meta-Llama-3-8B-Instruct, reaching accuracy $=0.22$0 and Macro-F1 $=0.22$1. In domain-name disputes, the best stronger-party Macro-F1 is achieved by Meta-Llama-3 under $=0.22$2, with accuracy $=0.22$3 and Macro-F1 $=0.22$4. In both domains, $=0.22$5 consistently exceeds $=0.22$6 and $=0.22$7 in stronger-party Macro-F1, indicating that explicit argument-wise reasoning improves party-level resolution assistance (Pawar et al., 2 Sep 2025).

Task and domain	Best setup	Result
Stronger party, $=0.22$8	$=0.22$9 ensemble	Acc 0.80, Macro-F1 0.78
Stronger party, $=0.63$0	$=0.63$1 Llama-3	Acc 0.72, Macro-F1 0.62
Demand decisions, $=0.63$2	$=0.63$3 Mistral	Acc 0.63, Macro-F1 0.64
Demand decisions, $=0.63$4	$=0.63$5 ensemble	Acc 0.63, Macro-F1 0.64
Argument strength, $=0.63$6	$=0.63$7 ensemble	Acc 0.59, Macro-F1 0.60
Argument strength, $=0.63$8	$=0.63$9 GPT-4o-mini	Acc 0.71, Macro-F1 0.73

Demand-wise decision performance is lower than stronger-party performance but remains materially above the majority baselines. In $=0.40$0, the best demand-level Macro-F1 is $=0.40$1 under $=0.40$2 with Mistral. In $=0.40$3, the best demand-level Macro-F1 is $=0.40$4 under $=0.40$5 with the ensemble. This suggests that decomposing disputes into demand-specific accept/reject judgments is feasible but more difficult than identifying the stronger side globally.

Argument-wise evaluation is the hardest task in the insurance domain and comparatively stronger in domain-name disputes. In $=0.40$6, the best argument-strength result is the $=0.40$7 ensemble with accuracy $=0.40$8 and Macro-F1 $=0.40$9. In $=0.22$0, GPT-4o-mini under $=0.22$1 reaches accuracy $=0.22$2 and Macro-F1 $=0.22$3. A plausible implication is that the domain-name corpus offers argument patterns that are more regular or more legible to general-purpose LLMs than those in Indian automobile-insurance disputes.

The paper also evaluates justification text. GPT-4o-mini and Mistral generally achieve better similarity scores than Llama for stronger-party justifications. However, $=0.22$4 justifications score worse on ROUGE and BERTScore than $=0.22$5 or $=0.22$6, despite producing better classification performance. The reason given is that $=0.22$7 encourages a formulaic explanation pattern—stronger because one party’s arguments are strong and the other’s are weak—whereas the reference justifications are more case-specific in wording. The resulting drop in lexical or embedding similarity does not imply weaker decision accuracy.

6. Biases, limitations, and place in legal AI

Several limitations are explicit. First, the entire system is zero-shot: there is no task-specific fine-tuning, and all adaptation is through prompt design. Second, errors in structured summarization propagate into downstream resolution. If arguments are misattributed or incompletely summarized, later stages can reason over the wrong dispute structure. Third, demand-wise and argument-wise gold labels are created with GPT-4o-mini rather than human legal experts, which introduces model-dependent supervision noise (Pawar et al., 2 Sep 2025).

Bias is especially salient in the domain-name setting. Mistral predicts the complainant as the stronger party in almost all cases, and the other models also show some tendency in that direction. Even in the best $=0.22$8 Llama setting, recall for the respondent label is only $=0.22$9. This appears to reflect both the real class imbalance in WIPO decisions and model over-reliance on priors. The ensemble underperforms the best single model in $=0.62$0 partly because Mistral’s complainant bias contaminates majority voting.

The system’s evaluation remains limited in scope. Only two dispute domains are studied. There is no human trial measuring whether adjudicators using DRAssist become faster, more accurate, or more consistent. The authors also note that the structured summaries are evaluated only approximately, using rhetorical-role extraction rather than human gold summaries. These factors constrain how far one can generalize from the reported metrics.

Within the broader legal-AI landscape, DRAssist differs from conventional supervised judgment-prediction systems by making argument structure, demand structure, and explanation generation first-class outputs. Its novelty lies less in model architecture than in the combination of structured summarization, multi-level prompting, and task design. The system thereby illustrates a distinct direction in legal AI: not merely predicting who wins, but producing an inspectable intermediate representation of why a dispute may be resolved in a particular way (Pawar et al., 2 Sep 2025).