Multi-person Multi-charge Prediction (MPMCP)

• MPMCP is a legal prediction setting that determines defendant-specific outcomes from narratives involving multiple defendants and multiple charges.
• It operationalizes four scenarios (S1–S4) to capture challenges like multi-label charge assignment and defendant disambiguation in complex cases.
• The benchmark uses a Chinese dataset from real criminal judgments, revealing performance degradation as case complexity increases.

Multi-person Multi-charge Prediction (MPMCP) is a legal judgment prediction setting in which a model must predict the judicial outcome for a specific defendant from a case narrative that may involve multiple defendants, with each defendant facing one or multiple charges, together with the corresponding penalty term. In the benchmark introduced by "MultiJustice: A Chinese Dataset for Multi-Party, Multi-Charge Legal Prediction" (Wang et al., 9 Jul 2025), MPMCP is presented as a more realistic alternative to conventional legal judgment prediction (LJP) benchmarks, which often simplify criminal cases into effectively single-defendant, single-charge instances. The work’s central contribution is a dataset and benchmark framework, rather than a new architecture, for measuring how existing legal and general LLMs degrade as case structure becomes more entangled (Wang et al., 9 Jul 2025).

1. Conceptual scope and motivation

MPMCP denotes prediction over criminal cases where a factual description may contain multiple defendants, and each defendant may receive one or multiple charges plus a penalty term. The key distinction from standard LJP is that the prediction target is not merely a case-level outcome; it is a defendant-level outcome conditioned on a shared factual narrative (Wang et al., 9 Jul 2025).

The motivation is explicitly tied to the mismatch between benchmark simplifications and real judicial documents. Conventional LJP settings often assume a much simpler mapping from facts to outcomes, whereas real Chinese criminal judgments can involve co-defendants, distinct criminal acts, different legal responsibilities, and different penalties. The cited empirical background underscores this gap: MAMD contains about 30.32% multi-defendant cases, and MultiLJP reports that 89.58% of cases have defendants receiving different judgments for at least one subtask (Wang et al., 9 Jul 2025). This supports the claim that flattening cases into homogeneous single-actor, single-charge instances can erase legally consequential structure.

A common misconception is to treat MPMCP as merely a harder variant of multi-label classification. The benchmark argues for a more specific interpretation: the hard part is not only that a defendant may have multiple accusations, but also that the model must disentangle which acts, roles, and liabilities belong to which defendant within a shared narrative (Wang et al., 9 Jul 2025). This suggests that MPMCP combines label cardinality, entity disambiguation, and long-context legal reasoning.

2. Scenario taxonomy and prediction granularity

The benchmark operationalizes MPMCP through four practical scenarios:

$\text{S1}: \text{single defendant, single charge}$

$$\text{S2}: \text{single defendant, multiple charges}$$

$$\text{S3}: \text{multiple defendants, single charge}$$

$$\text{S4}: \text{multiple defendants, multiple charges}$$

These scenarios encode increasing structural complexity rather than merely increasing label count (Wang et al., 9 Jul 2025). S1 corresponds to the classical benchmark case: one defendant, one charge, one sentencing outcome. S2 introduces multi-label charge assignment for one defendant. S3 introduces defendant disambiguation, because the model must isolate one defendant’s liability from a factual narrative involving several participants. S4 combines both forms of difficulty simultaneously.

The paper emphasizes S4 as the most realistic and hardest setting. Its motivating contrast is between a simple theft case and an S4 case where two defendants should receive different outcomes from a narrative involving two charges and multiple activities or methods. The benchmark’s main empirical conclusion is summarized as:

$\text{S4} > \text{S2} > \text{S3} > \text{S1}$

where “$>$” means harder (Wang et al., 9 Jul 2025). However, this ordering is a broad empirical conclusion rather than a strict invariant across every model and metric. For example, in charge prediction, InternLM2 has 91.8 F1 in S2 and 81.0 in S3, which means S3 is harder than S2 for that model; for penalty prediction, Lawformer has 46.4 LogD in S2 and 48.7 in S3, again indicating a mixed pattern (Wang et al., 9 Jul 2025).

The prediction granularity is central. Inputs contain a Defendant field and the full Fact text; outputs are defendant-specific statements such as “Defendant B is charged with XX” or “Defendant B is sentenced to X Years X Months.” Thus, one case with multiple defendants can yield different labels for different defendants, even though all predictions use the same case-level factual narrative as context (Wang et al., 9 Jul 2025).

3. MultiJustice dataset and construction pipeline

The dataset underlying MPMCP is built from first-instance criminal judgment documents collected from China Judgments Online covering 1998 to 2021. The jurisdiction is China, the language is Chinese, and the retained documents are criminal judgments with clearly identifiable defendants, factual descriptions, charges, penalty terms, and applicable legal articles (Wang et al., 9 Jul 2025).

Construction relies on regular expressions over standard judgment sections identified by keyphrases such as “Upon trial, it was found”, “This court believes”, and “The judgment is as follows.” The first section, a basic case introduction, is not used for dataset construction. The sections used are the Facts section, the Reasoning / legal applicability section, and the Judgment section. The Facts section is used as model input; the Reasoning / legal applicability section contains the judge’s explanation and applicable legal articles; and the Judgment section contains per-defendant outcomes, including charges and prison terms (Wang et al., 9 Jul 2025).

A defining design choice is that the dataset preserves all defendants and their corresponding judgments for each case, rather than collapsing the case into one overall label. Extraction includes defendant identification, charge assignment, penalty term extraction, and multi-defendant / multi-charge representation via the scenario split S1–S4 (Wang et al., 9 Jul 2025). The paper does not provide a formal graph or tuple representation, does not define symbolic notation such as $x = \{d_i, c_i, p_i\}$, and does not formalize multi-defendant alignment mathematically beyond prose definitions of the four scenarios.

Several quality-control steps are reported. The authors mask any content in the factual text that exactly matches charge names, randomly select 5,000 cases for each of the four scenarios, and manually assess approximately 5% of the data (Wang et al., 9 Jul 2025). The resulting dataset contains 20,000 cases, balanced by design across scenarios:

Scenario	Cases	Share
S1	5,000	25%
S2	5,000	25%
S3	5,000	25%
S4	5,000	25%

The dataset includes 306 charges and 234 legal articles. The paper also reports #Term = 1, reflecting that penalty prediction is treated as a single sentencing outcome variable rather than a taxonomy of discrete term labels. With respect to scenario-level distributions, the number of charges is predominantly concentrated around 1–2 charges; S1 and S3 involve only 1 charge per case; S2 and S4 have an average of 2 charges per case; and there are outliers ranging from 3 to 10 charges (Wang et al., 9 Jul 2025). The paper does not provide a full frequency table for charge imbalance or exact numeric descriptive statistics for penalty-term distributions.

For each setting, the split is 4,000 training, 500 validation, and 500 testing cases, yielding 16,000 train, 2,000 dev, and 2,000 test across the four scenarios if aggregated (Wang et al., 9 Jul 2025).

4. Task formulation, prompts, and evaluation

The benchmark evaluates two LJP tasks: charge prediction and penalty term prediction (Wang et al., 9 Jul 2025). Charge prediction is described as a standard classification task, although in S2 and S4 it is effectively a multi-label outcome because one defendant may have multiple charges. The paper does not clarify whether implementation uses exact-set classification over charge combinations or independent multi-label decisions; it reports standard classification metrics only.

The prompt templates make the input-output format explicit. Inputs contain a Defendant field naming the target defendant and the Fact text extracted from the judgment. Outputs are textual statements in one of three forms: a charge statement, a penalty statement, or a joint multitask statement such as “Defendant A is charged with XX, and sentenced to X Years X Months” (Wang et al., 9 Jul 2025). This design reinforces that the benchmark is defendant-level rather than pure case-level prediction.

For charge prediction, the reported metrics are:

$\text{Accuracy}(\%) \uparrow$

$\text{Precision}(\%) \uparrow$

$\text{Recall}(\%) \uparrow$

$$\text{S2}: \text{single defendant, multiple charges}$$0

For penalty term prediction, the benchmark uses:

$$\text{S2}: \text{single defendant, multiple charges}$$1

The paper describes LogD as measuring the logarithmic difference between the predicted penalty term and the ground truth value, with lower values better (Wang et al., 9 Jul 2025). It does not provide the formula for LogD. Likewise, it does not provide formulas for Accuracy, Precision, Recall, or F1, and it does not specify whether F1 is macro, micro, or weighted. A plausible implication is that evaluation is performed over defendant-level prediction instances, given the input-output design, but this is not stated explicitly.

The task formalization is therefore intentionally pragmatic rather than mathematically explicit. The paper supplies no formal objective function, no loss in equations for MPMCP itself, and no structured prediction formalization over defendants, charges, and penalties (Wang et al., 9 Jul 2025).

5. Baselines, training settings, and empirical behavior

The benchmark compares five open-source models: MT5, mBERT (reported as BERT in the results table), RoBERTa, Lawformer, and InternLM2 (Wang et al., 9 Jul 2025). Among them, Lawformer is a long-document legal model based on Longformer and pre-trained on Chinese legal case documents, while InternLM2 is based on internlm2-base, additionally pre-trained on domain-specific corpora, and is presented as the strongest model in the benchmark for charge prediction.

The authors report that they fine-tuned benchmark models using the per-scenario train/dev/test splits. For InternLM2, they additionally study supervised fine-tuning, multi-task learning, prompting without examples, and prompting with one example. Retrieval for the in-context example uses a BM25 retriever to select the most similar case from the test setting and append it as a demonstration example (Wang et al., 9 Jul 2025). The paper does not specify context lengths, truncation rules, decoding parameters, constrained decoding, post-processing rules, or hyperparameters such as learning rate, batch size, epochs, optimizer, or hardware.

The main empirical pattern is that performance degrades as scenario complexity increases. For charge prediction, InternLM2 achieves the highest F1 in every scenario: 85.3 in S1, 91.8 in S2, 81.0 in S3, and 80.8 in S4. By contrast, Lawformer records 81.4, 72.4, 78.0, and 61.7 across the same scenarios, indicating a much sharper drop in the hardest setting (Wang et al., 9 Jul 2025).

For penalty term prediction, the reported LogD values are more mixed. Lawformer attains 39.5 in S1, 46.4 in S2, 48.7 in S3, and 58.5 in S4, which matches the claim that harder scenarios lead to larger sentencing-distance errors. InternLM2 records 59.3, 54.1, 61.3, and 56.5 across S1–S4 (Wang et al., 9 Jul 2025).

The paper highlights two model-specific comparisons from S1 to S4. For Lawformer, the degradation is internally consistent with the results table: about 19.7% lower F1-score and 19.0% higher LogD. For InternLM2, the paper states approximately 4.5% lower F1-score and 2.8% higher LogD in S4 compared to S1 (Wang et al., 9 Jul 2025). However, the reported table gives 59.3 LogD in S1 and 56.5 in S4, which is numerically lower rather than higher. The most faithful interpretation is that the paper’s prose and table are inconsistent on this specific comparison.

The InternLM2 ablation further indicates that supervised fine-tuning on each subtask gives the best overall performance, multi-task learning underperforms fine-tuning, and adding one retrieval-based demonstration example improves prompting over zero-shot prompting in all scenarios (Wang et al., 9 Jul 2025). On charge prediction, the fine-tuning F1 values are 85.3 / 91.8 / 81.0 / 80.8 for S1–S4, while prompting without example yields 59.7 / 67.3 / 62.3 / 57.8 and prompting with one example improves these to 65.0 / 82.4 / 73.4 / 68.4. On penalty prediction, the corresponding LogD values are 59.3 / 54.1 / 61.3 / 56.5 for fine-tuning and 105.6 / 83.8 / 103.1 / 84.0 for prompting without example (Wang et al., 9 Jul 2025).

6. Relation to prior work, limitations, and research implications

MPMCP is closely related to, but distinct from, earlier work on multi-label charge prediction. "An External Knowledge Enhanced Multi-label Charge Prediction Approach with Label Number Learning" (Wei et al., 2019) addresses the problem of predicting multiple accusations for a single case/document and introduces a Number Learning Network (NLN) to predict how many labels should be output. Its formulation is explicitly case-level: one fact description in, one set of charges out. It does not define defendant-level prediction, does not distinguish multiple persons within one case, and has no mechanism for assigning different charge sets to different defendants (Wei et al., 2019).

This makes the relation between the two lines of work precise. The 2019 paper overlaps with MPMCP only in the multi-charge dimension. It is relevant as a reference for multi-label charge selection, adaptive label-number prediction, and external knowledge from law provisions, but not as a direct solution to the multi-person problem (Wei et al., 2019). A plausible implication is that methods like NLN could be repurposed in MPMCP only after constructing defendant-specific representations and defendant-specific charge score vectors.

The 2025 MPMCP benchmark also identifies several limitations and open issues. It notes that S4 cases have longer factual descriptions, greater variability in facts, legal articles, and sentencing outcomes, and more compositional and contextual variance (Wang et al., 9 Jul 2025). The paper does not provide a dedicated error-analysis section with confusion matrices or formal failure taxonomies, but its scenario reasoning implies several recurrent failure sources: confusion among defendants sharing one narrative, entanglement between acts and liabilities, long-context understanding demands, and difficulty predicting a specific defendant’s sentence when multiple acts and charges are involved.

The benchmark’s practical implications are correspondingly structural. Future systems should model defendant-specific reasoning within shared narratives, support multi-label charge prediction jointly with sentencing, handle long legal contexts robustly, improve generalization across scenario complexity, and ideally provide better interpretability (Wang et al., 9 Jul 2025). On the data side, benchmarks should preserve all defendants and all charges per case, avoid leakage by masking charge-name mentions in facts, include balanced evaluation across scenario types, and retain realistic structural complexity instead of collapsing cases.

The paper also records several caveats for deployment: limited generalizability beyond Chinese criminal law, possible dataset bias, fairness concerns, privacy risks despite anonymization, and lack of interpretability in LLMs (Wang et al., 9 Jul 2025). These issues place MPMCP not only as a benchmark extension but as a more exacting test of whether legal AI systems can preserve legally relevant distinctions that simplified LJP settings systematically obscure.