Meaningful evaluation and comparison of microarchitectural side-channel attacks

Determine how microarchitectural side-channel attacks should be meaningfully evaluated and compared, including which benchmarks and metrics provide valid, reproducible, and attack-relevant assessments.

Background

The paper surveys benchmarking practices for microarchitectural side-channel attacks and argues that commonly used proxies—such as covert-channel bandwidth or key recovery against outdated cryptographic implementations—may not accurately characterize the capabilities of a new primitive. Comparisons are further complicated by differences in hardware, software, protocols, threat models, noise conditions, and experimental setups.

Although the paper proposes a taxonomy of benchmarking flaws, resolution metrics, and recommendations for fairer evaluations, it does not establish a generally accepted methodology or unified benchmark for meaningfully evaluating and comparing these attacks. The authors therefore present the issue as an unanswered foundational question for the field.