Cause of Qwen3-30B-A3B verification variability across implementations and hardware

Determine whether the observed differences in Token-DiFR and Activation-DiFR behavior for Qwen3-30B-A3B across inference implementations (vLLM versus HuggingFace Transformers) and GPU types (H200 versus A100) are driven primarily by the model’s mixture-of-experts architecture or by implementation-specific factors in the inference stacks, in order to clarify the source of variability that affects detector calibration and deployment.

Background

Across models, Token-DiFR generally performs consistently; however, the Qwen3-30B-A3B results show substantially more variability than Llama-3.1-8B and Qwen3-8B. The largest shifts arise when switching between vLLM and HuggingFace implementations, and noticeable changes appear when moving between H200 and A100 GPUs within vLLM.

When verifier and provider match (H200/vLLM), Qwen3-30B-A3B achieves very high exact-match rates (~99.9%) and minor misconfigurations are detected reliably. In pooled settings including multiple implementations, minor deviations become harder to detect, suggesting that understanding whether architecture (mixture-of-experts) or implementation details dominate is important for practical verification and calibration.

References

It is unclear whether these differences are driven primarily by the mixture-of-experts architecture or by implementation details in the current inference stacks.

— DiFR: Inference Verification Despite Nondeterminism  (2511.20621 - Karvonen et al., 25 Nov 2025) in Appendix K, Qwen3-30B-A3B Results

A controlled attribution study remains future work rather than a condition of the protocol claim, and it is the study that would turn Ablation~A's endpoint contrast into a route claim. Its preregistered design holds the Qwen3.8-27B checkpoint fixed across five cells: pinned vLLM BF16, SGLang BF16, vLLM FP8, hosted AkashML BF16, and the retained self-hosted RunPod endpoint. It reports score, contract-valid response rate, contract executability and GPU-hours over IB-6 Core or a fixed stratified 64-task sample.

— IBIB: A Protocol for Measuring Enterprise AI Systems by Serving Route, Not Model Identifier  (2609.10494 - Stenstrom et al., 9 Sep 2026) in Section 8.1, “The study needed for run-to-run and engine attribution”

We have not replicated on a second serving stack (e.g.\ SGLang), so we cannot say which findings are vLLM-specific.

— Interface-Induced Trajectory Censoring  (2609.03966 - Wang, 3 Sep 2026) in Section 4, Limitations, item 15