Python-calibrated stress configurations for recovering concurrency flakiness

Determine whether stress-ng configurations specifically tuned for Python's execution model recover concurrency-induced flaky tests that the default configurations inherited from Java and Android fail to detect.

Background

Shaker's stress workloads were transferred unchanged from a study of Java and Android systems and had not been calibrated for Python. The present evaluation found no significant advantage over plain re-execution, but the authors acknowledge that this result may reflect a mismatch between the inherited stress settings and Python's execution model rather than an inherent inability of stress-based detection to expose concurrency-related flakiness.

References

Three questions our data could not settle remain open: whether detection improves materially as the execution budget grows past the $\sim$170-rerun threshold of \citeauthor{gruber2021empirical}; whether stress-ng configurations tuned for Python's execution model, rather than inherited from Java and Android, recover the concurrency cases the default configuration misses; and whether running each test within its suite, rather than in isolation, restores part of the flakiness that did not reproduce here.

— Evaluating Shaker for Flaky Test Detection in Python Projects  (2609.25528 - Leal et al., 22 Sep 2026) in Section Conclusions