A machine-learning-assisted progressive digit-randomness screening framework for detecting non-random patterns in raw numerical research data

Abstract: Raw numerical datasets remain less systematically examined in integrity screening than images, plagiarism, or summary-statistic inconsistencies. We developed the Fabrication-risk Digit Randomness Screening model (FDRS), a statistical and machine-learning framework for detecting non-random digit-pattern irregularities in numerical research data. FDRS integrates single- and joint-decimal-digit tests, Cramer's V, entropy metrics, Kullback-Leibler divergence, digit-preference indices, progressive subsampling, and semi-supervised risk scoring. It was evaluated using an instrument-derived enzymatic absorbance dataset (RawData, n=253) and a blinded manually simulated irregular dataset (ErrData, n=255). RawData showed no significant deviation in single third-decimal-digit analysis, whereas ErrData showed a significant deviation. In joint third-fourth decimal digit analysis, ErrData showed higher Cramer's V, lower normalized entropy, higher KL divergence, and a more persistent progressive-subsampling deviation signal. In internal validation, Elastic-net Logistic Regression achieved the highest AUC (0.98395) and lowest Brier score (0.048439), while Random Forest achieved the highest accuracy (0.926667) and balanced accuracy (0.935). RawData received a low ensemble risk score of 0.124627 and was classified as Grade 0; ErrData received a score of 0.740760 and was classified as Grade 3. External real-world benchmarks supported graded risk stratification: three datasets without identified public post-publication concerns were classified as Grade 0 or 1, whereas two datasets from publicly questioned or institutionally handled articles were classified as Grade 2 or 3. FDRS can prioritize raw numerical datasets for further review by integrating interpretable statistical and machine-learning features. It is an auxiliary digit-structure screening tool, not standalone evidence of fabrication or misconduct.

• The paper introduces a rigorous FDRS framework that blends statistical digit-distribution tests with machine learning to flag potential data fabrication.
• It employs progressive subsampling and metrics like normalized entropy and KL divergence to distinguish genuine measurement variability from manipulation.
• Validation on synthetic and real-world datasets demonstrates strong classification performance, underscoring its potential for scalable research data integrity screening.

A Machine-Learning-Assisted Progressive Digit-Randomness Screening Framework for Non-Random Pattern Detection in Raw Numerical Research Data

Introduction and Motivation

The integrity of raw numerical research data is fundamental to scientific reproducibility and reliability, yet current data integrity screening practices in the biomedical and broader scientific community are concentrated on image manipulation, plagiarism, duplicate publication, and anomalies in summary statistics. Methods for the systematic analysis of raw numerical data for potential fabrication or manipulation remain underdeveloped and lack both methodological rigor and broad adoption. The paper “A machine-learning-assisted progressive digit-randomness screening framework for detecting non-random patterns in raw numerical research data” (2606.07128) introduces the Fabrication-risk Digit Randomness Screening model (FDRS), a comprehensive framework that combines statistical testing with machine learning for the detection of non-random patterns in the terminal digits of raw numerical research data.

Methodological Framework

FDRS is predicated on the statistical expectation that, under plausible measurement and recording procedures, certain decimal digits—particularly in the third and fourth places—should approximate a uniform distribution if they arise from genuine, continuous experimental measurements captured at sufficient numerical resolution. Deviations from this expectation, notably in the context of digit preference or joint digit combinations, may flag human intervention such as fabrication, selective rounding, or excessive manual adjustment.

Statistical Feature Engineering

FDRS integrates multiple statistical tests and information-theoretic metrics in a structured pipeline:

• Single- and Joint-Decimal Digit Analysis: The framework employs multinomial goodness-of-fit and joint-distribution tests on specific decimal positions (primarily third and fourth) to evaluate digit randomness.
• Measures of Deviation: Effect sizes (Cramér’s V), normalized Shannon entropy, and Kullback-Leibler (KL) divergence quantify the magnitude and nature of deviations from the theoretical uniform digit distribution.
• Digit-Preference Indices: A range of explicit indices (e.g., 0/5-preference, odd/even ratios, repeated-value index) are calculated to reflect known human digit selection tendencies.
• Progressive Subsampling: Rather than relying on potentially unstable full-sample statistics, FDRS examines the stability of deviation as a function of increasing sample size. Metrics are tracked across subsamples to distinguish stochastic small-sample anomalies from persistent structural irregularity.

Machine Learning and Risk Scoring

A unique aspect of FDRS is its semi-supervised machine learning module due to the scarcity of authentic, confirmed-fabrication datasets. Training utilizes simulated normal datasets (from uniform, normal, log-normal, and gamma distributions), digit-preference contaminated datasets, and mixed-contamination datasets that emulate realistic partial fabrication.

• Supervised Classifiers: Random Forest, Elastic-net Logistic Regression, and RBF-kernel SVM are used to learn discriminative patterns in the engineered feature space, with a focus on both interpretability and performance.
• Anomaly Detection: Isolation Forest, primarily trained on reference-normal data, offers a complementary anomaly detection perspective, targeting broader departures from reference digit structure.
• Integrated Ensemble Risk: Final risk scoring leverages an ensemble approach, fusing classifiers and anomaly detectors, and assigns datasets to predefined risk grades for triage.

Experimental Validation

Synthetic and Real-World Datasets

The system was rigorously validated on multiple types of datasets:

• Proof-of-Concept Comparison: Instrument-measured absorbance data (RawData, $n=253$) versus manually simulated pseudo-experimental data (ErrData, $n=255$). RawData closely followed expected digit distributions (third-decimal $\chi^2=7.91$, $p=0.5433$, Cramér’s $V=0.0589$, $H^*=0.9932$) while ErrData showed significant deviation (third-decimal $\chi^2=17.59$, $p=0.0403$, $V=0.0875$, $H^*=0.9851$), consistent with subtle but systematic human digit preference.
• Joint Digit Distribution and Progressive Subsampling: In joint third-fourth decimal analysis, neither dataset violated uniformity by classical $n=255$0 thresholds, but ErrData systematically exhibited higher effect sizes and lower entropy. Critically, progressive subsampling revealed RawData’s deviation attenuated with larger samples, whereas ErrData’s deviation persisted or increased.
• Machine Learning Classification: In semi-supervised cross-validation, Elastic-net Logistic Regression delivered AUC $n=255$1 and Brier score $n=255$2, Random Forest achieved accuracy $n=255$3, balanced accuracy $n=255$4, specificity $n=255$5, and F1 $n=255$6. Ensemble risk scoring for RawData was $n=255$7 (Grade 0), while ErrData was $n=255$8 (Grade 3), corresponding to no apparent and strong irregularity, respectively.

Real-World Benchmarking

The framework was benchmarked on five independent, published datasets: three without known post-publication integrity concerns (RealRawData1-3) and two associated with formal data integrity actions (RealErrData1-2). All RealRawData* sets were scored as Grade 0 or 1 (risk scores: $n=255$9); RealErrData sets received significantly higher risk scores ($\chi^2=7.91$0 and $\chi^2=7.91$1; Grades 2 and 3). This ordinal concordance supports the framework’s value as a risk-stratification/prioritization signal, while highlighting the non-binary nature of digit-pattern irregularities in heterogeneous real data.

Implications and Theoretical Significance

FDRS addresses a salient gap in quantitative data integrity screening, particularly for domains where numerical data do not span orders of magnitude and Benford-type analysis is inapplicable. The explicit focus on terminal digit randomness, combined with a multi-metric and progressively-stabilized approach, enables the identification of anomalies uniquely reflective of manual data manipulation or digit preference in laboratory and biomedical data.

Practically, the adoption of FDRS holds potential for automated, high-throughput pre-publication or post-publication screening pipelines, institutional research integrity offices, and funding agency audits, complementing existing image and text-based approaches. The interpretability of features and attribution to specific digit patterns increases transparency and auditability in research integrity investigations.

Theoretically, the work positions digit-structure analysis as a legitimate subdomain of statistical forensics, opening avenues for further linking micro-structural numerical anomalies to specific classes of research misbehavior. The progressive subsampling methodology could be generalized to other domains of high-dimensional integrity screening where sample size and statistical fluctuation interact nonlinearly.

Limitations and Future Directions

The framework is constrained by critical dependency on simulated irregular references due to the paucity of confirmed-fabrication datasets. As a result, spectrum coverage of potential irregularity mechanisms (copy-paste artifacts, variance compression, local duplication) may be incomplete. Risk scores are generated at the dataset level, lacking granularity for flagging individual anomalous values. Real-world validation was necessarily limited in size and heterogeneity, and benchmark categories cannot be considered definitive ground truth.

Future expansion should include:

• Large-scale, multi-platform authentic raw data acquisition.
• Diversified simulation (e.g., copy-paste blocks, range compression, over-randomization).
• Enhanced interpretability (e.g., SHAP, likelihood-ratio frameworks).
• Fine-grained risk attribution and per-value anomaly explanation.
• Calibration of risk thresholds and false positive rates against larger, adjudicated benchmarks.

Conclusion

The FDRS framework represents a methodologically rigorous, multi-metric statistical and machine learning approach for the prioritized screening of raw numerical research data. By integrating digit-distribution tests, information-theoretic metrics, progressive sampling stability, and semi-supervised risk modeling, it provides a transparent, auditable signal for datasets with non-random digit patterns. Strong classification performance and ordinal risk-stratification on real-world benchmarks reinforce its practical applicability. However, outputs must not be construed as direct evidence of fabrication or misconduct; rather, they serve to prioritize further expert review and verification—a critical advance toward systematic, scalable numerical data integrity screening.