Detecting speculative leaks with compositional semantics

Abstract: Speculative execution enhances processor performance by predicting intermediate results and executing instructions based on these predictions. However, incorrect predictions can lead to security vulnerabilities, as speculative instructions leave traces in microarchitectural components that attackers can exploit. This is demonstrated by the family of Spectre attacks. Unfortunately, existing countermeasures to these attacks lack a formal security characterization, making it difficult to verify their effectiveness. In this paper, we propose a novel framework for detecting information flows introduced by speculative execution and reasoning about software defenses. The theoretical foundation of our approach is speculative non-interference (SNI), a novel semantic notion of security against speculative execution attacks. SNI relates information leakage observed under a standard non-speculative semantics to leakage arising under semantics that explicitly model speculative execution. To capture their combined effects, we extend our framework with a mechanism to safely compose multiple speculative semantics, each focussing on a single aspect of speculation. This allows us to analyze the complex interactions and resulting leaks that can arise when multiple speculative mechanisms operate together. On the practical side, we develop Spectector, a symbolic analysis tool that uses our compositional framework and leverages SMT solvers to detect vulnerabilities and verify program security with respect to multiple speculation mechanisms. We demonstrate the effectiveness of Spectector through evaluations on standard security benchmarks and new vulnerability scenarios.

• The paper introduces a compositional framework integrating Speculative Non-Interference (SNI) to formally model and verify speculative leaks across multiple prediction mechanisms.
• It employs an innovative always-mispredict semantics to abstract prediction behaviors, enabling detection of composite leaks missed by isolated analyses.
• Spectector, the implemented symbolic program analyzer, leverages SMT solvers to automatically verify security guarantees against memory and control-flow leaks.

Formal Framework for Speculative Leak Detection Using Compositional Semantics

Introduction and Motivation

The paper "Detecting speculative leaks with compositional semantics" (2603.29800) addresses the persistent challenge of security vulnerabilities in modern processor architectures due to speculative execution. Speculative execution, a cornerstone of performance optimization in contemporary CPUs, enables out-of-order and predictive instruction processing. However, it introduces non-trivial microarchitectural side-effects, notably exploited by Spectre-class attacks, where traces left by speculatively executed (and later squashed) instructions can be probed through side-channels like caches. Existing mitigation strategies are often ad hoc and lack rigorous formal guarantees, and most prior analyses focus only on isolated speculation mechanisms, failing to capture interactions between multiple predictors or speculative mechanisms found in real CPUs.

Semantics and Security Notions

To address this gap, the authors propose a compositional framework for modeling, reasoning, and verifying information flows induced by speculative execution across multiple prediction mechanisms. At the core of the framework is the Speculative Non-Interference (SNI) security property, which constrains speculative information flow by requiring that speculative traces do not reveal more than what is leaked by the non-speculative (architectural) semantics. SNI is parameterized both by a security policy distinguishing sensitive and public components (on memory and registers), and by the speculative semantics capturing particular prediction behaviors. The formal language used (μASM) abstracts x86-like assembly, supporting compositional operational semantics for different speculation sources (branches, jumps, stores, returns, straight-line speculation).

A notable advance is the introduction of an always-mispredict semantics (AM), which abstracts away specific prediction behaviors and enables reasoning about possibility of speculative leaks independent of microarchitectural implementation details. The AM model exhaustively explores the worst-case scenario for information leakage due to speculation, providing oracle-agnostic guarantees.

Compositionality of Speculative Semantics

The framework's compositionality is realized by defining operational and symbolic semantics for each predictor (branch, store-bypass, RSB, indirect-jump, SLS), and then providing rules for their safe composition. This enables the detection of speculative leaks that only arise from the interaction of multiple speculation mechanisms—a substantial theoretical refinement over prior work that hard-codes single mechanisms.

Composed semantics rely on a metaparameter $Z$, which determines instruction classes delegated to each component semantics. Well-formedness conditions (confluence, projection preservation, state relation, symbolic preservation) ensure soundness and maximal reuse of proofs between source and composed semantics. This design allows formal guarantees (overapproximation and consistency) to propagate automatically to their compositions.

Symbolic Program Analysis Tool: Spectector

The theoretical developments are realized in a symbolic program analyzer called Spectector, implemented atop the CIAO logic programming system. Spectector takes x86 assembly, translates it to μASM, and applies symbolic execution under both non-speculative and compositional speculative semantics. By leveraging SMT solvers (Z3), it checks for violations of SNI via constraints reflecting divergent speculative traces with respect to the policy, thus automating the detection or verification of absence of speculative leaks.

Spectector systematically explores program paths by symbolic execution, using self-composition to encode pairs of runs in its satisfiability checks. This approach enables detection of both memory and control-flow leaks, and provides completeness and soundness with respect to the SNI property, due to the proven theoretical equivalence between symbolic and concrete semantics.

Empirical Evaluation and Results

Evaluation comprises two main case studies. Firstly, Spectector is assessed against standard microbenchmarks for Spectre-PHT, Spectre-STL, Spectre-BTB, Spectre-RSB, and Spectre-SLS vulnerabilities. The tool reliably detects all known leaks and can verify patched code by compiler countermeasures (lfence, speculative load hardening, retpoline). Notably, the tool exposes subtle leaks missed by prior pattern-matching symbolic analyses—e.g., Spectre-PHT with conditional moves. The symbolic AM semantics supports detection regardless of optimization level or mitigations.

Secondly, the paper demonstrates the necessity and effectiveness of compositional semantics by constructing examples where leaks only arise from the interaction of multiple mechanisms (e.g., branch + store, branch + jump + return). Spectector can detect such "composite leaks"—which are missed by single-mechanism analyses—validating the compositional approach.

Implications and Future Directions

The implications are direct for both software and hardware security. Practically, the framework and tool provide a rigorous basis for static verification of code against speculative execution leaks, supporting compiler and microarchitecture design with quantitatively precise security guarantees.

Theoretically, the compositional semantics method establishes a scalable, extensible foundation for side-channel analysis. As new speculation mechanisms are discovered or introduced, the framework can accommodate them without extensive reengineering of proof obligations or tool implementations.

Areas of further research include:

• Extending the framework to concurrency and multi-core contexts, where speculative leaks may propagate across thread or process boundaries.
• Incorporating more granular microarchitectural models (e.g., cache types, TLBs) into the observer definition, trading off precision and tractability.
• Generalizing to hardware-assisted security mechanisms (shadow stacks, memory tagging, secure enclaves), possibly enforcing SNI at the hardware-software contract level.

Conclusion

In summary, the paper presents a comprehensive formal and practical solution for detecting and verifying speculative leaks in processor-executed programs. By introducing the compositional semantics framework, the authors unify modeling, analysis, and verification of speculative information flows, covering both isolated and composite speculation mechanisms. The symbolic tool Spectector achieves strong soundness and completeness guarantees, demonstrated on standard and newly crafted benchmarks, and sets the stage for future advances in speculation-aware secure compilation and architecture design.