EnsemJudge: Ensemble Detection for Chinese LLM Text

• EnsemJudge is an ensemble meta-detector for Chinese LLM-generated text, combining rule-based, training-free, supervised, and LoRA-tuned models.
• Its dynamic voting mechanism and strategy assignment module adaptively weight signals based on text length, domain, and adversarial cues.
• By leveraging Chinese-specific linguistic features and robust evaluation protocols, EnsemJudge achieves superior macro F1 scores in challenging scenarios.

EnsemJudge is an ensemble-based framework for detecting Chinese LLM-generated text, designed and evaluated in the NLPCC2025 Shared Task 1. It combines heterogeneous detectors—rule-based heuristics, training-free methods, supervised classifiers, and LoRA-tuned Chinese LLMs—through a strategy assignment module, dynamic weighted voting, and optional LLM-based decision support, with the explicit aim of maintaining reliability under out-of-domain and adversarial conditions such as paraphrasing, mixed human–LLM text, and short inputs (Wang et al., 30 Mar 2026).

1. Scope, motivation, and problem formulation

EnsemJudge addresses the problem of distinguishing human-written from LLM-generated Chinese text. The motivating risks identified for this setting include academic cheating, large-scale fake news or misinformation, and mislabeled LLM content being treated as human opinion. The framework is designed for a deployment regime in which inputs are not limited to clean in-domain data, but include out-of-domain texts, adversarial paraphrases, mixed human–AI passages, and short excerpts, all of which can degrade single-detector performance substantially (Wang et al., 30 Mar 2026).

The framework is explicitly Chinese-centric. The underlying paper emphasizes Chinese-specific tokenization behavior, the absence of whitespace segmentation, Chinese punctuation and clause structure, and the limited availability of strong public Chinese benchmarks relative to English. It also notes that some methods effective on English degrade noticeably on Chinese, and that no single detector performs well across all Chinese text categories, including normal, short, paraphrased, and mixed cases (Wang et al., 30 Mar 2026).

Formally, EnsemJudge treats the decision problem as binary classification. For a text $t$, each base detector $M_i$ outputs a binary prediction

$v_{M_i}^t \in \{-1,1\},$

where the positive label denotes LLM-generated text and the negative label denotes human-written text. The overall system then combines these base predictions with text-dependent strategies and weights rather than relying on a fixed classifier (Wang et al., 30 Mar 2026).

This design suggests that EnsemJudge is best understood as a meta-detector: the central object is not a single model, but a mechanism for selecting, weighting, and reconciling multiple detection signals under distribution shift (Wang et al., 30 Mar 2026).

2. System architecture and dynamic voting mechanism

The EnsemJudge pipeline has five components: a collection of base models, a strategy assignment module, a dynamic voting module, a decision support module, and an optional decisive adjustment module. At inference time, the system first extracts lightweight features from the input text, uses those features to choose a strategy and associated weights, collects the selected base-model predictions, and then performs weighted voting. In uncertain cases, it can invoke a large LLM as an auxiliary arbiter (Wang et al., 30 Mar 2026).

The core voting score for a text $t$ is

$s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$

where $w_{M_i}^t$ denotes the weight assigned to model $M_i$ under the strategy selected for $t$. The final binary output is computed as

$$y^t = \begin{cases} 1, & \text{if } s^t + \lambda \cdot d^t \geq \tau \ 0, & \text{otherwise}, \end{cases}$$

where $d^t \in [-1,1]$ is the decision-support signal from an auxiliary LLM, $M_i$0 controls the strength of that signal, and $M_i$1 is the decision threshold (Wang et al., 30 Mar 2026).

The strategy assignment module is text-conditional. It extracts lightweight features such as length and perplexity, clusters the training texts using these features, and assigns to each cluster a specific subset of base models plus a weight vector. At test time, a new text is mapped to a cluster, and the corresponding strategy is applied. This makes the ensemble dynamic rather than static: the set of active detectors and their voting strengths depend on the input regime, especially whether the text is extreme short, short, medium, or general (Wang et al., 30 Mar 2026).

The decision support module is invoked when the ensemble is uncertain, operationalized as cases where the voting score is near threshold. It uses Qwen2.5-72B-Instruct with few-shot in-context learning, and the few-shot examples include back-translated texts so that the model learns to recognize paraphrased LLM outputs. The resulting support score $M_i$2 is blended into the final decision through the $M_i$3-weighted term in the decision rule above (Wang et al., 30 Mar 2026).

The decisive adjustment module is a pattern-based override mechanism for highly distinctive cues, such as double newlines that appear only in LLM outputs. The authors ultimately conclude that a strong ensemble makes this module largely unnecessary and potentially risky, because adversarial artifacts can make hard-coded overrides brittle (Wang et al., 30 Mar 2026).

3. Detector ecology and heterogeneity of the ensemble

A defining property of EnsemJudge is heterogeneity. Its base model pool deliberately mixes rule-based heuristics, training-free scoring methods, supervised encoders, and instruction-tuned Chinese LLMs, with the explicit goal of exploiting complementary error profiles rather than maximizing the standalone strength of any single family (Wang et al., 30 Mar 2026).

Detector family	Representative models	Primary role
Rule-based	SpecialToken, CommonPhrase, SentenceSegment, ConsecutivePunctuation	Capture niche surface artifacts
Training-free	Binoculars, Fast-DetectGPT, CommonToken	Zero-shot statistical detection
Training-based	Chinese BERT, Chinese RoBERTa, HybridFeatureRoBERTa	Supervised binary classification
LoRA-tuned Chinese LLMs	Qwen2.5-7B-Instruct, GLM-4-9B-Chat	Strong high-level semantic detection

The rule-based detectors are intentionally simple. SpecialToken checks special tokens such as \n\n; CommonPhrase tracks Chinese phrases disproportionately associated with LLM outputs; SentenceSegment inspects clause-unit patterns; ConsecutivePunctuation captures repeated punctuation correlated with human informality. Their standalone macro F1 scores are low, but the paper argues that they contribute complementary signals in edge cases (Wang et al., 30 Mar 2026).

The training-free detectors rely on distributional asymmetries rather than supervised fitting. Binoculars is adapted with Qwen2.5-7B as backbone. Fast-DetectGPT is used in both standard and analytical variants with Qwen2.5-7B, Qwen2.5-7B-Instruct, and GLM-4-9B-Chat. CommonToken is a self-designed detector: it tokenizes with the GLM-4-9B tokenizer, computes token frequencies separately on LLM and human subsets of the training data, and predicts according to which side’s frequent tokens dominate the text (Wang et al., 30 Mar 2026).

The training-based classifiers include Chinese BERT and Chinese RoBERTa fine-tuned for binary classification, plus length-specific variants trained on 55–75-word or 110–150-word text. HybridFeatureRoBERTa extends RoBERTa by concatenating hidden representations with outputs from rule-based and training-free detectors, then training a new classification head on the joint feature vector (Wang et al., 30 Mar 2026).

The strongest standalone components are the LoRA-tuned Chinese LLMs trained with LLaMAFactory, especially Qwen2.5-7B-Instruct-LoRA and its short-text variants. These models are instruction-tuned to classify whether a given text is LLM-generated, and they serve as the ensemble’s high-capacity semantic anchors under adversarial and out-of-domain conditions (Wang et al., 30 Mar 2026).

4. Robustness design: adversarial reliability, length awareness, and Chinese-specific adaptation

The paper gives EnsemJudge an explicit notion of reliability under adversarial transformation. Let $M_i$4 be a detection system, $M_i$5 a set of LLM-generated texts, and $M_i$6 a family of adversarial transformations. The transformed corpus is

$M_i$7

and reliability is defined as

$M_i$8

Because enumerating all $M_i$9 is infeasible, the implementation approximates this with a representative subset $v_{M_i}^t \in \{-1,1\},$0, centered on paraphrasing and excerpting attacks (Wang et al., 30 Mar 2026).

Two adversarial augmentation mechanisms are central. First, back-translation with mbart-large-50-many-to-many-mmt produces Chinese $v_{M_i}^t \in \{-1,1\},$1 English $v_{M_i}^t \in \{-1,1\},$2 Chinese paraphrases. Second, text excerpting creates shorter segments from longer passages. These augmentations are derived solely from the official training set, satisfying the shared-task restriction against external corpora, and they are used to probe which detectors remain stable when text form changes but authorship does not (Wang et al., 30 Mar 2026).

Length is treated as a first-class variable. The paper reports that all detectors improve on longer texts, with performance saturating around 500 characters, and that Qwen2.5-7B-Instruct-LoRA improves by 39.0%, 14.5%, and 1.1% as length increases across the reported bins. EnsemJudge therefore uses length-specific models, length-specific thresholds for Binoculars and Fast-DetectGPT, and separate ensemble weight vectors for “extreme short,” “short,” “medium,” and “general” cases (Wang et al., 30 Mar 2026).

The framework is also explicitly adapted to Chinese linguistic structure. Training-based detectors use Chinese BERT and Chinese RoBERTa; LLM backbones include Qwen2.5 and GLM-4; CommonToken uses the GLM-4-9B tokenizer; and rule-based features target Chinese punctuation and sentence segmentation. The paper argues that these design choices are not interchangeable with English-oriented defaults, and that Chinese-optimized models such as Qwen2.5 are more robust under paraphrasing than many English-derived detection methods (Wang et al., 30 Mar 2026).

This suggests that EnsemJudge’s robustness is not solely an effect of “more models.” It also depends on the deliberate coupling of adversarial augmentation, length-aware specialization, and language-specific feature design (Wang et al., 30 Mar 2026).

5. Dataset, evaluation protocol, and empirical performance

EnsemJudge is trained and evaluated on the dataset of NLPCC2025 Shared Task 1, with DetectRL-ZH as the training source. The split summarized in the paper is: Train $v_{M_i}^t \in \{-1,1\},$3 LLM / $v_{M_i}^t \in \{-1,1\},$4 human, Dev $v_{M_i}^t \in \{-1,1\},$5 LLM / $v_{M_i}^t \in \{-1,1\},$6 human, and Test $v_{M_i}^t \in \{-1,1\},$7 LLM / $v_{M_i}^t \in \{-1,1\},$8 human. The test set is deliberately out-of-domain in both content domain and generating LLM source, and it includes adversarial subsets—Mixed, Paraphrase, and Perturbation—as well as varying-length subsets at 64, 128, 256, and 512 characters (Wang et al., 30 Mar 2026).

The primary shared-task metric is macro F1-score. On the full test set, EnsemJudge achieves 0.9922 macro F1. The subset breakdown reported in the paper is: Normal $v_{M_i}^t \in \{-1,1\},$9, Mixed $t$0, Paraphrase $t$1, Perturbation $t$2, and by length, 64 $t$3, 128 $t$4, 256 $t$5, 512 $t$6 (Wang et al., 30 Mar 2026).

The margin over individual base detectors is substantial. Representative full-test macro F1 values are: SpecialToken $t$7, SentenceSegment $t$8, CommonToken(GLM-4-9B) $t$9, Fast-DetectGPT variants $s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$0–$s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$1, Binoculars $s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$2, ChineseBERT $s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$3, ChineseRoBERTa $s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$4, HybridFeatureRoBERTa $s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$5, GLM-4-9B-Chat-LoRA $s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$6, Qwen2.5-7B-Instruct-LoRA $s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$7, and Qwen2.5-7B-Instruct-LoRA (Short Text) $s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$8. EnsemJudge’s $s^t = \sum_{i=1}^{n} w_{M_i}^t \cdot v_{M_i}^t,$9 therefore exceeds the strongest single model by roughly 3.5 points (Wang et al., 30 Mar 2026).

The robustness analysis is especially revealing. On the Mixed subset, the best single model reported is Qwen2.5-7B-Instruct-LoRA (Short Text) at $w_{M_i}^t$0, whereas EnsemJudge reaches $w_{M_i}^t$1. On Paraphrase, Qwen2.5-7B-Instruct-LoRA reaches $w_{M_i}^t$2, while EnsemJudge reaches $w_{M_i}^t$3. On Perturbation, EnsemJudge reaches $w_{M_i}^t$4, matching the best standalone result. The paper also shows that training-free methods are especially fragile under attack—for example, Binoculars drops from $w_{M_i}^t$5 on Normal to $w_{M_i}^t$6 on Mixed, $w_{M_i}^t$7 on Paraphrase, and $w_{M_i}^t$8 on Perturbation—whereas Qwen2.5-7B-Instruct-LoRA remains comparatively stable (Wang et al., 30 Mar 2026).

The short-text regime is another major test. Qwen2.5-7B-Instruct-LoRA achieves only $w_{M_i}^t$9 at length 64, but its Extreme Short variant rises to $M_i$0. EnsemJudge, by combining length-aware strategies, reaches $M_i$1 at 64 characters. This is one of the clearest demonstrations that the framework’s dynamic strategy assignment, rather than mere model averaging, is central to its performance (Wang et al., 30 Mar 2026).

6. Position within ensemble-judge research and open limitations

Despite its name, EnsemJudge belongs to a different lineage from response-quality “LLM-as-a-judge” systems. It is a detector of Chinese LLM-generated authorship, not a benchmark or reward model for evaluating candidate answers. This places it closer to ensemble meta-detection than to systems such as JudgeBlender, which ensembles prompts and small LLMs for automatic relevance assessment (Rahmani et al., 2024), Auto-Prompt Ensemble, which learns auxiliary evaluation dimensions and aggregates them through Collective Confidence (Li et al., 8 Oct 2025), or AudioJudge, which decomposes speech assessment into lexical, paralinguistic, and quality judges (Manakul et al., 17 Jul 2025).

At the same time, EnsemJudge shares a family resemblance with those systems in its reliance on heterogeneity, conditional aggregation, and specialized subcomponents. A plausible implication is that it can be read as a concrete demonstration of a broader principle also visible in multilingual and multimodal judge research: complementary failure modes can be exploited if the aggregation mechanism is sufficiently structured. SEMJ, for example, treats cross-lingual inconsistency as a useful signal for iterative refinement rather than mere noise (Fu et al., 6 Jun 2026), while judge-aware ranking frameworks model judge-specific reliability rather than treating all evaluators equally (Xu et al., 29 Jan 2026). EnsemJudge adopts the same spirit in a different domain by assigning different subsets of detectors and different weights to different text regimes (Wang et al., 30 Mar 2026).

The paper nonetheless identifies clear limitations. The framework is heavily tuned to the NLPCC2025 Shared Task 1 setting, and may require retuning for other datasets or domains. Its components are Chinese-specific, including tokenization, pretrained backbones, and rule patterns, so cross-lingual transfer is not established. Strategy assignment, cluster formation, and weight tuning are largely manual or heuristic rather than learned end-to-end. The adversarial space remains partial, focused on paraphrase, excerpting, mixing, and perturbation rather than more complex style-transfer or human-edit attacks. Finally, the decision support module relies on Qwen2.5-72B-Instruct, which is computationally expensive even if only invoked for uncertain cases (Wang et al., 30 Mar 2026).

These limitations do not diminish the paper’s central result. EnsemJudge shows that, for Chinese LLM-generated text detection under strong out-of-domain and adversarial shift, reliability is not well served by a single detector family. It emerges instead from a deliberately engineered ensemble in which rule-based, probabilistic, supervised, and instruction-tuned components are combined through length-aware, cluster-specific, and uncertainty-sensitive voting (Wang et al., 30 Mar 2026).