- The paper defines nonrepudiation for experiments as a cryptographic mechanism that binds reported results to actual computations, ensuring tamper-evidence.
- It presents K-Veritas, a Go-based testbed that wraps training runs to record runtime telemetry and securely attest experimental metrics without major pipeline changes.
- A phased adoption plan is outlined to make nonrepudiable attestations mandatory, aiming to deter fraud and enhance credibility across computational sciences.
Nonrepudiable Experimental Results as a Foundation for Computational Science
Motivation and Problem Analysis
The paper "Computer Science Conferences Should Require Nonrepudiable Experimental Results" (2605.08586) advances a rigorous position on experimental verification in computational fields, highlighting the persistent and structural shortcomings in current reproducibility approaches. The reproducibility crisis in ML and broader computational sciences has been repeatedly demonstrated by failed replication attempts, fabrication, and non-verifiable claims, as summarized through surveys and large-scale analyses [13, 17, 20]. Measures like self-reported checklists, artifact evaluation, and experiment logging platforms have improved transparency marginally but, crucially, remain voluntary, author-controlled, or insufficiently binding to the actual computations reported.
The authors formally identify the problem as one of experiment nonrepudiation: the absence of protocols that bind reported experimental results to the computation that produced them in a manner that is tamper-evident and author-independent, precluding later denial or alteration of the results. They rigorously distinguish nonrepudiation from reproducibility, provenance, and artifact availability, arguing that no current mechanism robustly verifies the authenticity of the presented numbers.
Nonrepudiation, as redefined for computational experiments, entails the existence of a cryptographically signed and tamper-evident attestation that:
- Binds reported metric values to a concrete execution, capturing executable code, configuration, hardware fingerprint, and telemetry.
- Ensures that the attestation is generated and signed by an entity or service independent of the author, with the author lacking access to the private signing key.
- Satisfies properties of passivity, data blindness, tamper-evidence, and independent verifiability.
Through this definition, the security model explicitly considers attacks not merely as accidental or administratively induced but as intentional fabrications—including text-level falsification, log modification, selective reporting, and the construction of “fake” computational evidence. Critically, the protocol’s limits and attack surface are described: while software-only approaches cannot mitigate a fully privileged (OS-kernel or hardware-level) adversary, they drastically raise the cost and skill required for fraudulent claims compared to the prevailing status quo.
K-Veritas: A Practical Testbed
To demonstrate tractability, the authors introduce K-Veritas, a reference implementation in Go meeting their abstract criteria for compliant nonrepudiation protocols. K-Veritas operates as a user space binary acting as a passive observer: it wraps arbitrary training runs, collects hashes of source files, records runtime telemetry (CPU, GPU, memory, disk I/O), parses metric output, and generates a single canonical digest. This digest is sent to a remote signing service, whose cryptographic signature (RSA-PSS-SHA256) is returned, ensuring author-key separation. Reports and archives generated by the tool are independently verifiable, and the design precludes exposure of raw data or metric trajectories, complying with data sensitivity concerns. The framework demonstrates that nonrepudiation properties can be achieved in real machine learning pipelines without significant architectural changes.
While K-Veritas is not immune to kernel- or firmware-level compromise, it is sufficient to prevent straightforward fabrication and text-level denial of computation. The authors further propose a hardware-metric consistency (HMC) heuristic—linking hardware activity to declared claims—as a further check against shallow or fake runs.
Threat Models, Limitations, and Adoption Path
The authors forcibly enumerate the attack vectors covered and not covered by their framework. Unprotected classes include attacks leveraging subverted kernels, device firmware, or compromised attestation servers. The latter is addressed through operational controls and federated governance, a model borrowed from critical open infrastructure (e.g., software signing ecosystems).
Operational limitations are transparently acknowledged: the protocol does not itself verify experimental design quality—it only asserts that results originated from real, author-unalterable computation. It also requires author compliance, necessitating institutional buy-in. Full deployment at conference scale requires robust infrastructure for attestation, rate limiting, session storage, and auditing.
For adoption, a phased model is proposed:
- Phase 1: Voluntary, with attested reports receiving visible recognition.
- Phase 2: Expected, with automated checks and reviewer integration.
- Phase 3: Mandatory, making nonrepudiable attestations a precondition for empirical publication.
This sequence mirrors trajectories previously seen with artifact evaluation and code sharing guidelines, anticipating a multi-year adoption period.
Implications and Future Directions
If adopted at scale, the proposal would enforce a fundamental shift in the evidentiary standard for empirical claims in computer science and ML. Papers would no longer be judged simply on the plausibility and clarity of results, but also on formally verifiable evidence that the results were actually generated by the claimed code and configuration.
Practical implications include:
- Deterrence of Casual Fabrication: Raising the technical bar for fraudulent results from mere document editing to the necessity of executing real computation, absent ability to compromise attestation systems.
- Enhanced Credibility and Accountability: Honest researchers obtain stronger evidentiary support; bad actors face greater barriers to misconduct.
- Generalization Across Fields: While instantiated with ML, the proposal is universalizable to any empirical science reliant on computational pipelines (e.g., systems benchmarking, computational science, agent evaluation).
- Governance Considerations: Attestation infrastructure must avoid centralization risks through independent, federated operation, with transparent protocols and public auditing.
Theoretically, this work paves the way for future integration of hardware-backed attestation (e.g., TEEs) to address the small class of privileged adversaries. In combination with pre-registration and artifact evaluation, nonrepudiation protocols offer a blueprint for a robust, multimodal evidence chain for empirical scientific claims.
Conclusion
The authors make a clear, forceful case that the computer science community must move beyond voluntary and self-reported reproducibility compliance, toward cryptographically underpinned nonrepudiation of experimental results. By precisely defining required properties, constructing explicit threat models, and providing a viable testbed implementation, they lay a practical and conceptual foundation for a future in which empirical papers are inseparable from auditable, nonrepudiable claims. While not a panacea, nonrepudiation represents an essential building block for restoring and maintaining trust in the scientific record, with direct implications across all computational sciences.
