Bona Fide Cross-Testing Methods

• Bona fide cross-testing is an evaluation approach that uses independent, untampered test settings to ensure reliable assessment and avoid contamination.
• It employs methods like Cross-Context Verification and hierarchical architectures to compute diversity, contamination scores, and detect systematic biases.
• The technique applies across domains such as model contamination, biometric spoofing, and audio deepfake detection, emphasizing the importance of independent evaluation conditions.

Bona fide cross-testing denotes a class of testing and evaluation procedures in which credibility depends on crossing genuinely independent contexts rather than reusing a single contaminated context or collapsing heterogeneous conditions into one aggregate score. In one recent usage, benchmark contamination detection is recast as session-isolated replication of the same task across independent API conversations, so that repeated convergence indicates recall rather than feedback leakage (Song, 23 Mar 2026). In another established usage in biometric and audio anti-spoofing, bona fide refers to genuine, untampered inputs, and cross-testing evaluates whether a detector trained under one data regime generalizes to unseen bona fide distributions, unseen attack generators, or both (Paulo et al., 28 Jan 2026, Kwok et al., 11 Sep 2025).

1. Independence as the defining criterion

A central distinction in the literature is between genuine cross-testing and repeated analysis within a single informational stream. In benchmark contamination work, the critical requirement is that each solve be an independent observation of model behavior: “a fresh API conversation with no prior history,” “a clean repository clone at the problem’s base commit,” “future git history purged,” and “identical prompts containing only the issue text.” The aim is to ensure that later trials cannot inherit intermediate artifacts from earlier ones. This is why Cross-Context Verification (CCV) is described as a legitimate cross-test: it compares behavior across separate contexts that cannot leak information to one another, much like independent experimental replications (Song, 23 Mar 2026).

An analogous concern appears in audio deepfake detection. Standard evaluation often combines one bona fide subset with many spoof subsets and reports a single pooled Equal Error Rate (EER), but the pooled threshold is determined over the entire combined distribution, so larger subsets dominate the threshold while smaller ones can be washed out. Bona fide cross-testing is proposed precisely to vary the bona fide side as well, using multiple bona fide datasets crossed with multiple synthesizer subsets, thereby exposing vulnerabilities that a pooled EER obscures (Kwok et al., 11 Sep 2025).

This shared emphasis on independence also clarifies a common misconception. Bona fide cross-testing is not merely “repeat the same question” or “evaluate on more data.” The crucial property is that the crossed conditions remain analytically meaningful: isolated sessions in contamination detection, unseen bona fide datasets in anti-spoofing, or separately planned half-studies in observational inference.

2. Cross-Context Verification and hierarchical isolation

In benchmark contamination detection, CCV solves the same benchmark problem $N$ times in independent sessions and measures the variation among the resulting patches $p_1,\ldots,p_N$. Pairwise similarity is computed from AST structural similarity with weight $0.4$, line-level BLEU with weight $0.3$, and normalized edit distance with weight $0.3$. Diversity is then defined as the complement of average pairwise similarity:

$\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$

The paper notes that the typesetting appears to omit a closing parenthesis or a slash in the denominator, but the intended meaning is the complement of average pairwise similarity. CCV also computes gold proximity as the mean and standard deviation of BLEU similarity between each trial patch and the gold patch,

$$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$

and combines these into a contamination score,

$$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$

with thresholds HIGH $\geq 0.8$, MEDIUM $\geq 0.6$, and LOW $p_1,\ldots,p_N$0 (Song, 23 Mar 2026).

The same paper adds a reasoning classifier as a direct behavioral signal. It inspects the first 100 tokens and assigns NO_REASONING if the output begins with ```diff or patch content and total tokens are fewer than 300, and FULL_REASONING if the output begins with analytical phrases such as “Looking at,” “The issue is,” or “Let me analyze.” The methodological claim is that direct observation of whether the model emits a patch immediately or begins with analysis is more diagnostic than paraphrase consistency, n-gram overlap, or perplexity analysis, because those methods do not directly observe whether the model reasons or recalls (Song, 23 Mar 2026).

The analysis pipeline is then extended into the Hierarchical Cross-Context Architecture (HCCA), a multi-agent framework whose key mechanism is information restriction rather than role proliferation.

Layer	Role	Function
Layer 0	Designer	Creates the protocol and assigns groups, but never touches the experimental data
Layer 1	Executor	Runs solve_in_isolation() and collects raw patches, metrics, and logs
Layer 2	Analysts	Three independent analysts, each assigned a different group of problems
Layer 3	Integrator	Combines Layer 2 results, cross-validates them, resolves discrepancies, and computes final statistics

The specific design principle is that Layer 2 analysts see only Layer 1 data and not one another’s conclusions. The paper presents this as a defense against anchoring and confirmatory interpretation, and explicitly contrasts it with Dynamic Cross-Context Review, where multi-turn review with the same role generates false positives faster than it discovers true errors (Song, 23 Mar 2026).

3. Empirical behavior, thresholds, and negative results

On 9 SWE-bench Verified problems comprising 45 trials with Claude Opus 4.6 at temperature 0, CCV is reported to achieve perfect separation between contaminated and genuine reasoning, with Mann-Whitney $p_1,\ldots,p_N$1, exact one-tailed $p_1,\ldots,p_N$2, and effect size $p_1,\ldots,p_N$3. The contaminated scores are $p_1,\ldots,p_N$4, $p_1,\ldots,p_N$5, and $p_1,\ldots,p_N$6; the genuine scores are $p_1,\ldots,p_N$7, $p_1,\ldots,p_N$8, $p_1,\ldots,p_N$9, $0.4$0, $0.4$1, and $0.4$2. Every contaminated score exceeds every genuine score, and the threshold of $0.4$3 cleanly divides the two groups, with the only intermediate case being the contamination-flaw composite (Song, 23 Mar 2026).

The paper frames several findings around this result. First, contamination is described as binary in the reported sample: the two confirmed contaminated problems show zero diversity, gold BLEU above $0.4$4, very short response times, and no reasoning preamble, whereas the non-contaminated cases do not. Second, the reasoning classifier is reported to achieve 100% accuracy on the dataset, perfectly separating contaminated-behavior trials from genuine-reasoning trials. Third, 33% of prior contamination labels are identified as false positives: astropy-13236, previously labeled suspicious, is reclassified as genuine reasoning with a moderate diversity score and response behavior similar to the control group. Fourth, test flaws are said to be separable from contamination, but not always cleanly; astropy-7606 is presented as a contamination-flaw composite with contamination score $0.4$5, discovered because one analyst identified a pure test flaw while another, working independently, detected contamination as well (Song, 23 Mar 2026).

The most important negative result is the pilot Worker-Verifier-Director experiment. In that setup, a Worker generates findings, a Verifier sees the findings and labels them confirmed or rejected, and a Director makes final decisions. Because downstream stages can see upstream conclusions, the Verifier confirms 100% of findings, the Director accepts 100% of those confirmations, and there is no filtering benefit at all. The paper labels this sycophantic confirmation and uses it to argue that information restriction, not structural complexity, is the operative mechanism (Song, 23 Mar 2026).

The reported limitations are correspondingly narrow and explicit: only 9 problems are tested; only Claude Opus 4.6 is evaluated; the reasoning classifier is evaluated on the same data it helps analyze; the test-flaw score can degenerate when diversity is near zero, requiring a correction; and HCCA adds computational overhead and is most useful in ambiguous cases. This suggests that the positive separation result is methodologically striking but not yet a claim of broad cross-model universality.

4. Face morphing detection and the bona fide class

In single-image morphing attack detection (S-MAD), bona fide denotes genuine, untampered face images, while morph attacks are images produced by blending or synthesizing two identities so that they appear acceptable to face recognition systems. FD-MAD formulates the problem as distinguishing morph $0.4$6 from bona fide $0.4$7, and argues that morphs differ from bona fide samples in the frequency domain, especially in the mid-to-high frequencies (Paulo et al., 28 Jan 2026).

The cross-testing protocol in FD-MAD is explicitly cross-dataset and cross-morph. Training is performed exclusively on SMDD, with the same number of bona fide images and morphs; global features are classified by an SVM with RBF kernel, local region classifiers use logistic regression, and local aggregation is performed by MRF fusion. Testing then proceeds on FRLL-Morph and MAD22, with an additional evaluation on MorDIFF. In this setup the bona fide test images in FRLL-Morph and MAD22 are unseen with respect to SMDD, so the bona fide class itself is cross-dataset rather than only the morph class (Paulo et al., 28 Jan 2026).

The method models the natural spectral decay of an image by

$0.4$8

and defines the residual

$0.4$9

Residual features are extracted globally and from the left eye, right eye, nose, and mouth. Region-level fusion is defined through unary potentials

$0.3$0

and a fully connected pairwise MRF,

$0.3$1

with local and fused scores

$0.3$2

where $0.3$3 (Paulo et al., 28 Jan 2026).

Performance is reported with APCER, BPCER, EER, and BPCER at fixed APCER operating points. Trained on SMDD and tested on FRLL-Morph, the method achieves average EER $0.3$4, BPCER@APCER=$0.3$5 of $0.3$6, and BPCER@APCER=$0.3$7 of $0.3$8. On MAD22, the corresponding values are average EER $0.3$9, BPCER@APCER=$0.3$0 of $0.3$1, and BPCER@APCER=$0.3$2 of $0.3$3. Classical attacks on FRLL-Morph are separated extremely well, whereas StyleGAN2 is harder; on MAD22, WebMorph and MIPGAN-II are particularly challenging (Paulo et al., 28 Jan 2026).

A related study on incremental synthetic non-morphed images examines two cross-dataset directions, FERET $0.3$4 FRGCv2 and FRGCv2 $0.3$5 FERET, while progressively adding synthetic bona fide samples from SMDD at 10%, 20%, 30%, 50%, 75%, and 100%. It reports that controlled amounts of synthetic bona fide data can improve cross-dataset generalization, but indiscriminate use can be suboptimal, and training only with synthetic data yields the highest EER in both directions. In FERET $0.3$6 FRGCv2, the best D-EER occurs around 50–75% addition; in FRGCv2 $0.3$7 FERET, the best D-EER occurs at 10% addition and larger additions often worsen performance. Synthetic-only training produces D-EERs near 38% in both directions (Benavente-Rios et al., 10 Oct 2025). A plausible implication is that bona fide cross-testing in S-MAD is sensitive not only to attack realism but also to how the bona fide distribution itself is constructed.

5. Audio spoofing and deepfake detection

In synthetic voice spoofing countermeasure systems, the relevant cross-testing problem is not only unknown attacks but also bona fide channel mismatch. A cross-dataset study trains on ASVspoof2019LA training, validates on ASVspoof2019LA development, and evaluates on ASVspoof2019LA evaluation, ASVspoof2015 evaluation, and VCC2020 using 60-dimensional LFCCs and EER as the performance metric. All three examined countermeasure systems degrade sharply cross-dataset: LCNN from $0.3$8 on 2019LA-eval to $0.3$9 on 2015-eval and $\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$0 on VCC2020; ResNet from $\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$1 to $\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$2 and $\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$3; ResNet-OC from $\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$4 to $\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$5 and $\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$6. The reported diagnostic finding is that spoofed trials are mostly still correctly scored, whereas the bona fide score distribution shifts downward substantially on ASVspoof2015 and VCC2020, causing false alarms. Average magnitude spectra of bona fide utterances differ across datasets, and a controlled channel-shift experiment using ASVspoof2019LA-Sim reproduces similar degradation, supporting the claim that channel mismatch in bona fide speech is a major cause of cross-dataset failure (Zhang et al., 2021).

The same study proposes three channel-robust strategies built on channel-augmented data: AUG, MT-AUG, and ADV-AUG. The multi-task objective is

$\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$7

and the adversarial objective with Gradient Reversal Layer is

$\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$8

$\text{diversity} = 1 - \frac{1}{\binom{N}{2} \sum_{i<j} \text{sim}(p_i, p_j)$9

Cross-dataset results improve substantially: on ASVspoof2015-eval, the vanilla ResNet-OC score of $$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$0 falls to $$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$1 with AUG and $$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$2 with ADV-AUG; on VCC2020, $$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$3 falls to $$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$4, $$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$5, and $$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$6 for AUG, MT-AUG, and ADV-AUG respectively (Zhang et al., 2021).

A more explicit formalization appears in bona fide cross-testing for audio deepfake detection. There, $$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$7 bona fide datasets $$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$8 are crossed with $$\bar{g} = \frac{1}{N}\sum_i \text{BLEU}(p_i, p_\text{gold}), \quad \sigma_g = \text{std}(\cdot),$$9 spoof/synthesizer datasets $$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$0, yielding $$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$1 evaluation settings and $$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$2 EERs. The bona fide-specific false positive rate is

$$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$3

and the pair-specific EER is

$$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$4

Because reporting every pair is unwieldy, the paper introduces maximum pooling over the spoof dimension,

$$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$5

so that each bona fide condition is associated with its worst-case synthesizer. The benchmark covers more than 150 synthesizers, 164 synthesizers in the reported results table, and nine bona fide speech types, including AMI-IHM, AMI-SDM, LibriSpeech test-clean, LibriSpeech test-other, VCTK, FakeAVCeleb-v1.2, In-The-Wild, EmoFake-EN, and AV-Deepfake-1M. The main reported finding is that models may miss more than 30% of spoof audio in the worst-case synthesizer setting even though average EER is around 10%, and that bona fide type matters substantially, with harder performance on FakeAVCeleb-v1.2 and AV-Deepfake-1M (Kwok et al., 11 Sep 2025).

6. Statistical and software analogues

Outside anti-spoofing and benchmark contamination, several adjacent literatures instantiate the same structural intuition: valid cross-testing requires separate realizations whose interaction is controlled rather than blended. In nonparametric statistics, the crossmatch test is a graph-based two-sample goodness-of-fit procedure for $$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$6 and $$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$7, testing $$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$8 against $$\text{CS} = 0.5\bar{g} + 0.3(1 - \text{diversity}) + 0.2(1 - \sigma_g),$$9. After pooling the samples and constructing a minimum-distance matching, the statistic $\geq 0.8$0 counts cross-sample matched pairs, and the paper proves that

$\geq 0.8$1

for matchings satisfying the stated conditions. Under the null the limit is $\geq 0.8$2; under any fixed alternative it is strictly smaller, yielding consistency against all fixed alternatives (Arias-Castro et al., 2015).

In observational studies with many hypotheses, cross-screening splits the sample at random into two halves, uses each half to plan the analysis of the other half, tests each half at level $\geq 0.8$3, and rejects by union,

$\geq 0.8$4

The paper proves strong family-wise error control,

$\geq 0.8$5

and emphasizes that replicability is not required: concurrence between halves is stronger, but either half alone can support rejection after the Bonferroni correction (Zhao et al., 2017).

In software testing, cross-coverage testing runs a test $\geq 0.8$6 generated for one functionally equivalent program $\geq 0.8$7 on another equivalent program $\geq 0.8$8, $\geq 0.8$9, and defines cumulative cross-coverage augmentation by adding useful suites from all other equivalent programs to a common initial suite. On 336 Java programs in 140 groups, cumulative cross-coverage improves coverage for 292 programs (86.90%), with average gains of about 26% statement coverage and about 33% branch coverage, and detects missing functionality in 124 of 169 valid modified programs (73.37%) (Bertolino et al., 2023). A related compiler-testing methodology compares the observable outputs of a compiler under test against outputs from preexisting tools such as GCC in C99 mode and the ARM ADS assembler; the reported outcome is several hundred bugs in the compiler toolchain, 21 correctness bugs in the assembler, 49 bugs in PalmSource’s modifications to FDLIBM, and 18 bugs in GCC, along with platform-specific GCC defects (Sheridan, 2022).

Taken together, these literatures suggest a common principle. Bona fide cross-testing is strongest when the crossed entities are genuinely independent or genuinely distinct—isolated sessions, unseen bona fide datasets, random half-samples, equivalent implementations, or external toolchains—and when the protocol prevents one condition from inheriting the latent conclusions of another. Where that independence is absent, the reported failure modes are correspondingly direct: pooled EERs become uninformative, repeated review generates false positives, downstream verifiers become sycophantic, and synthetic-only or single-corpus training can yield poor generalization.