Architectural Isolation as a Timing Safety Primitive for Edge AI Medical Devices: Controlled Experimental Evidence on a Shared-Silicon Platform

Abstract: A system can satisfy accuracy-based validation, maintain output stability (Safety-Threshold Exceedance Rate, STER, equal to zero), and still violate timing constraints under deployment load. These are structurally independent properties that current pre-market validation protocols often do not operationalize at the inference layer. This letter demonstrates their independence through a controlled same-hardware experiment: identical MobileNetV2 models are evaluated under identical adversarial load on two execution paths of the same NVIDIA Jetson Orin Nano Super, a dedicated GPU accelerator (TensorRT FP16, half-precision floating point) and a general-purpose CPU (ONNX Runtime FP32, single-precision floating point). Both paths maintain STER = 0; the CPU path (ONNX Runtime FP32) degrades 7.2x under combined load (mean latency 9.8x higher than the GPU path (TensorRT FP16), which maintains latency below 11 ms), breaching the 10 Hz clinical cycle budget by 65%. Joint STER and latency verification is proposed as a candidate method for operationalizing U.S. FDA Draft Guidance FDA-2024-D-4488 robustness requirements at the inference layer, subject to regulatory review and clinical validation.

• The paper demonstrates that architectural isolation can ensure zero Safety-Threshold Exceedance Rate while timing constraints are violated under realistic load.
• It employs a rigorous same-hardware experiment comparing GPU (TensorRT FP16) and CPU (ONNX Runtime FP32) inference paths on an NVIDIA Jetson Orin Nano Super.
• The study reveals a 9.8× mean latency gap under stress, emphasizing the necessity for joint output and timing validations in clinical AI applications.

Architectural Isolation as a Timing Safety Primitive in Edge AI Medical Devices

Summary and Problem Statement

This letter presents a controlled, same-hardware experimental study to investigate the independence of inference output stability and timing safety when deploying edge AI models in medical environments. The core finding is that a system can satisfy accuracy-based validation and maintain perfect output stability (Safety-Threshold Exceedance Rate, STER, equal to zero), while simultaneously violating critical timing constraints under realistic deployment-induced load. This study operationalizes key components of FDA Draft Guidance FDA-2024-D-4488 robustness requirements at the inference layer.

The experimental results are derived from observing MobileNetV2 executing on an NVIDIA Jetson Orin Nano Super in two configurations: (1) a GPU-accelerated path (TensorRT FP16) with architectural isolation and dedicated QoS bandwidth, and (2) a CPU-bound path (ONNX Runtime FP32) with no such isolation, competing for system resources with adversarial stressors. Both pipelines are evaluated under identical deployment stressors, revealing a 9.8× mean latency gap, with the CPU path breaching a representative 10 Hz clinical cycle deadline by 65% while the GPU path remains invariant.

Experimental Design

The methodological innovation is a rigorous same-hardware, split-path experimental protocol, which eliminates hardware-level confounds by controlling all system variables aside from the execution stack. The Jetson Orin Nano Super serves as the platform with both inference paths operating on shared LPDDR5 memory but diverging in isolation strategy: the GPU leverages dedicated DMA channels with bandwidth guarantees, while the CPU path directly contends with system stressors for bus and cache resources.

Figure 1: The experimental platform architecture—GPU inference (TensorRT FP16) benefits from dedicated DMA and system-level bandwidth reservation, while the CPU path (ONNX Runtime FP32) directly competes for bus and cache with stressors.

Test images (N=500) are processed under baseline and multiple stressor scenarios: CPU saturation, memory pressure, network I/O load, and a realistic combined scenario. Key metrics recorded per-inference include end-to-end latency and output deviation. STER is formally defined as the fraction of activations where the softmax vector deviates by more than $T^* = 0.05$ (infinity norm) from the zero-load reference, with output stability and timing quantified independently.

Empirical Results

Output Stability and Timing Orthogonality

Crucially, both paths maintained STER = 0 across all test conditions, certifying that the full output distributions never exceeded the prescribed deviation threshold, even under adversarial deployment loads. This finding holds for all 107,500 inference activations, precluding both hardware-induced and scheduling-induced output corruption.

However, latency measurements reveal an orthogonal failure mode: GPU latency remains constant at 10.6 ms mean and 10.9 ms $P_{99}$, while the CPU path exhibits mean latency swelling to 104.0 ms and $P_{99}$ rising to 165.1 ms under combined stress—well in excess of the 100 ms clinical time budget for 10 Hz operation. This timing failure is systematically undetected by any accuracy-based or output-stability-only metric.

Figure 2: Empirical CDFs of inference latency under baseline, CPU saturation, and combined stress. GPU latency is invariant; CPU path exceeds the clinical deadline under deployment load, despite no observable accuracy or output drift.

Implications for Pre-Market Validation

A key result is that a manufacturer performing accuracy-only evaluation would incorrectly certify both stacks as equivalent—and as safe—because classification accuracy ($\Delta\text{acc} < 1\%$) is unaffected by load. Existing protocols cannot detect timing safety violations unless latency is explicitly characterized under deployment-representative stressors. This exposes a validation gap: output correctness and timing reliability are structurally independent, and their joint verification is a necessary condition for regulatory acceptance.

Figure 3: STER remains zero across all experimental conditions and paths, while CPU path latency breaches clinical timing budgets—demonstrating independence between output stability and timing safety.

Discussion and Theoretical Implications

The study is distinguished by methodological rigor: maintaining identical hardware and varying only execution stack (TensorRT vs. ONNX; FP16 vs. FP32) isolates the impact of architectural isolation and resource scheduling on timing safety without confounding output stability. This yields a strong, replicable result: resource contention leads to timing failures, not silent output corruption, within the deterministic execution paradigm tested.

The work introduces and operationalizes joint STER + latency verification as an actionable safety metric for the inference layer in SaMD, aligning with IEC 62304 (software lifecycle) and ISO 14971 (risk management). It highlights the limitations of top-1 (argmax) based accuracy inspections, demonstrating that STER detects subthreshold distributional drift unobservable via legacy clinical metrics. Appropriate STER thresholds ($T^*$) must be determined through risk analysis per intended clinical use and validated for each device class.

Practical Implications and Future Directions

The empirical evidence supports the adoption of architectural isolation (e.g., dedicated accelerators with explicit bandwidth reservation) as a timing safety primitive in edge AI medical devices. For deployment on shared-silicon platforms, explicit timing validation—not just accuracy or output stability certification—is indispensable. Manufacturers and regulatory bodies must extend pre-market validation to cover both axes jointly.

Precision in inference (FP16 vs. FP32) and the runtime library (TensorRT vs. ONNX) are additional factors influencing absolute timing, but they do not influence the principal finding regarding the orthogonality of timing and output safety properties.

Future developments should extend this analysis to more complex pipelines, non-deterministic runtimes, real-time OS contexts, and clinical workflows with multi-model interactions. Additionally, clinical validation is mandatory to calibrate STER thresholds for safety-critical deployments. The methodology and scripts provided offer a reproducible framework for such extensions.

Conclusion

This study demonstrates, via controlled experimentation, that timing safety and output correctness are independent, orthogonal dimensions in edge AI inference—neither inferrable from the other. Existing accuracy or output-stability-only validation protocols are demonstrably insufficient for regulatory acceptance of edge AI in SaMD where timing budgets must be strictly maintained. The formal joint STER and latency condition is advanced as a candidate primitive for inference-layer robustness verification in medical AI, subject to regulatory validation and device-specific clinical thresholds.