Standardized evaluation methodology for blockchain detection systems

Establish standardized evaluation methodologies for blockchain-based intrusion-detection and response systems using common datasets, latency-under-load measurements, and consistent accuracy-versus-overhead trade-off curves.

Background

The paper identifies substantial inconsistency in how surveyed systems are evaluated. Many report very high detection accuracy on synthetic or simulated workloads while separately acknowledging significant processing and consensus overhead.

A standardized methodology would make it possible to compare detection quality, response latency, and computational overhead under realistic operating conditions rather than relying on isolated accuracy figures.

References

We flag evaluation-methodology standardization (common datasets, common latency-under-load reporting, and consistent accuracy/overhead trade-off curves rather than point estimates) as a cross-cutting open issue that would materially improve comparability across this literature.

— From Network Intrusion Detection to Blockchain-Backed Endpoint Detection and Response: Mapping the Landscape of Decentralized Detection-and-Response Architectures  (2610.01872 - Shahsavari et al., 1 Oct 2026) in Section 6.5, “Evaluation Methodology Weaknesses”