Robustness under stochastic decoding

Establish whether the Knowledge-Driven Analytics Framework's retrieval and generation results remain robust under stochastic decoding rather than only under deterministic temperature-zero decoding.

Background

All reported generation passes use temperature-zero decoding and produce byte-identical answers across seeds. This demonstrates reproducibility for the evaluated configuration but does not determine whether answer quality, citation traceability, or other measured properties persist when generation is sampled stochastically. The paper therefore identifies robustness under stochastic decoding as an untested issue and lists replication across random seeds as planned future work.

References

Our experiments use greedy decoding on English tasks; robustness across decoding strategies and temperatures is left to future work.

— ActMap: Single-Pass Uncertainty Quantification from Generation-Time Activation Maps  (2609.11498 - Dardini et al., 10 Sep 2026) in Limitations and Ethical Considerations

Fifth, decoding is deterministic at temperature 0, and the three generation passes reported here produced byte-identical answers. This establishes end-to-end reproducibility but is explicitly not evidence of robustness under stochastic decoding, which remains untested; readers should not interpret the absence of seed variance as stability under sampling.

— Auditable by Construction: An Ontology-Driven Framework for Trustworthy LLM Analytics in Enterprise Finance  (2608.20661 - Lunyakin, 21 Aug 2026) in Section 4.4, Limitations of the Pilot

We did not sweep temperature or top-$p$; whether the post-SFT Δ is robust to sampling stochasticity is untested.

— Procedural-skill SFT across capacity tiers: A W-Shaped pre-SFT Trajectory and Regime-Asymmetric Mechanism on 0.8B-4B Qwen3.5 Models  (2605.11907 - Strozzi, 12 May 2026) in Section 7, “Threats to Validity and Limitations,” item 7

Seed sensitivity. We fix one decoding seed (42) throughout, so individual outcomes are reproducible, but we have not measured how much a different seed would shift the reported rates.

— Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness  (2610.12361 - Sadhu et al., 8 Oct 2026) in Appendix, Section “Full Limitations,” paragraph “Seed sensitivity”