ShadowScope: GPU Kernel Monitoring

• ShadowScope is a GPU kernel execution monitoring framework that uses composable side-channel signals to validate runtime integrity.
• It employs instrumented kernels with modular markers and performance counter data to delineate and verify execution segments.
• The framework achieves high detection accuracy with minimal overhead through a segmented validation approach and optional hardware-assisted checks.

ShadowScope is a GPU kernel execution monitoring and validation framework that leverages composable side-channel signals to detect anomalous or potentially adversarial behavior in GPU kernels. Unlike traditional golden-model approaches that are sensitive to workload variation, noise, and interference, ShadowScope introduces a composable modeling paradigm, augmenting kernel execution with modular, repeatable functions to encode key behavioral features at finer granularity. It employs software-based monitoring using performance counters and a hardware-assisted variant, ShadowScope+, with on-chip checks for runtime validation at minimal system overhead (Almusaddar et al., 30 Aug 2025).

1. Framework Architecture and Operating Principles

ShadowScope provides continuous validation of GPU kernel integrity using side-channel observability. The core components are:

• Instrumented Kernels: Target kernels are instrumented with composable marker functions that delineate logical execution segments.
• Side-Channel Data Collector: Runtime metrics (e.g., instruction counts, memory loads/stores, atomic operations) are collected via the GPU Performance Monitoring Unit (PMU) using interfaces such as NVIDIA CUPTI.
• Composable Golden Model: Instead of monolithic reference traces, execution is decomposed into modular segments, each characterized by its own side-channel signature.
• Trace Validator: Captured traces are divided and aligned with their segment boundaries. Each segment's metrics are compared to a pre-recorded, trusted reference using statistical correlation or distance criteria.

This segmented approach mitigates the effects of intra-kernel scheduling variation and external noise, allowing ShadowScope to robustly infer integrity or detect kernel compromise.

2. Composable Modeling and Validation

The composable model is central to ShadowScope's robustness:

• Segment Definition: Execution is split via static markers—placed at kernel entry, exit, and critical boundaries—so each segment encodes a deterministic behavioral feature.
• Reference Matching: On validation, every segment’s signature is checked independently. Under the rule

$$\| \textbf{observed\_vector} - \textbf{golden\_vector} \| > \epsilon$$

for a set threshold $\epsilon$, a segment is flagged if the deviation exceeds tolerance. Only if multiple segments simultaneously deviate does the kernel flag as anomalous.

• Contextual Selection: Metadata about grid size and input shape is embedded with each marker, ensuring the verifier selects the matching golden trace for the current kernel configuration.

This modular design ensures that local variation (e.g., benign load balancing, OS jitter) in some segments does not induce global false alarms.

3. Use of Side-Channel Signals

ShadowScope repurposes hardware side channels, using them as signals for behavioral fingerprinting, rather than as attack vectors:

• PMU Events: Examples include instruction_executed, global_load, global_store, and global_atom_cas.
• Compositional Markers: Special atomic sequences (e.g., a loop of atomic compare-and-swap instructions) are used as markers, producing strong, isolated jumps in the associated hardware counters, enabling reliable segment boundaries even under noisy execution conditions.
• Granularity: Validation at the segment-level (rather than across the full kernel trace) improves resilience to noise and increases discriminative power for attacks like code injection, unexpected control-flow changes, or microarchitectural corruption (e.g., Rowhammer-induced faults).

The side-channel-based measurement is both processor-agnostic and implementation-agnostic, supporting portability across GPU architectures.

4. ShadowScope+: Hardware-Assisted Runtime Validation

ShadowScope+ extends the software monitoring approach with hardware support:

• Local PMUs per Streaming Multiprocessor (SM): Each SM maintains local performance counters, sampled over fine-grained windows, routed via a programmable multiplexer.
• On-Chip Validator: Performs aggregation and threshold-based comparison against onboard golden references, employing lightweight arithmetic (adders, comparators) for per-segment validation.
• Decision Logic: An anomaly is flagged if the per-segment metric difference exceeds $\epsilon$, as in the relation above.
• Real-Time Operation: Validation logic is integrated into the GPU interconnect, operating off the execution path, thus reducing latency and interference from the CPU, and enabling prompt response (e.g., kernel termination or alert generation) during a detected anomaly.

ShadowScope+ implementation yields <5% average runtime overhead ($4.6\%$) and negligible silicon area/power penalty (<0.03% area, <0.3% dynamic power), supporting its practicality in high-performance accelerator environments.

5. Scalability, Accuracy, and Deployment Implications

• Robustness: Experiments report 100% true positive rates (TPR) for detecting diverse memory or microarchitectural attacks, with low false positive rates, due to the modular verification strategy.
• Overhead: Runtime and resource costs are minimized since on-chip validation is pipelined and independent of the main kernel execution.

Technique	Runtime Overhead	Area/Power Overhead	Validation Granularity
Software-only	Moderate	None	Segment (PMU sample)
ShadowScope+ (HW)	4.6% avg	<0.03% area/<0.3% pwr	Per-sampling window

• Scalability: Composable segmentation supports kernel scaling, dynamic scheduling, and parallel kernel execution across large GPU arrays.
• Integration: Hardware changes (for ShadowScope+) require minimal architectural extensions. For software-only deployments, existing PMU and sampling interfaces are used, with care required to tune sampling rate and event groupings for the target platform.

6. Potential Limitations and Challenges

• Deployment on Commercial GPUs: Hardware integration (for ShadowScope+) necessitates vendor support, minor microarchitectural modifications, and updates to low-level toolchains.
• Noisy Environments: While the composable model is highly noise-tolerant, extreme concurrency, heavy multi-tenancy, or overlapping workloads may require further calibration of correlation/distortion thresholds.
• Threshold Tuning: Choice of $\epsilon$ is critical; if set too low, benign variability might be flagged as anomalous; too high, and subtle attacks may avoid detection.
• Legacy Support: On legacy hardware without on-chip support, validation accuracy and sampling frequencies are limited by PMU event grouping and system software constraints.

7. Application Domains and Impact

ShadowScope is directly applicable in scenarios where computational correctness and code/data integrity of GPU workloads are paramount, such as:

• Machine learning inference and training workloads (where silent memory or control-flow errors can affect model results).
• High-performance and cloud computing with strong tenant and isolation requirements.
• Scientific simulations sensitive to microarchitectural or soft errors.
• GPU-powered autonomous systems requiring runtime integrity guarantees.

By transforming side-channel observability from a liability into an active defense, ShadowScope demonstrates the feasibility of runtime GPU kernel validation at scale and with minimal system impact (Almusaddar et al., 30 Aug 2025).