Determine which effect scale exhibits greater temporal stability

Determine, using experiments with sufficiently large treatment effects, whether temporal stability is better represented by fixed log-odds contrasts or by fixed rate differences.

Background

The paper primarily defines persistence on the logit scale, where a fixed contrast corresponds to a fixed odds ratio. It also considers the rate-difference scale, noting that the two effect measures cannot both remain fixed when the common level changes. In the ASOS data, treatment effects were too small to distinguish the scales, so the authors explicitly leave the question unresolved and call for experiments with larger effects.

References

At a median absolute contrast of $0.0032$ the two scales give the same number to the precision the data offer, so this dataset cannot discriminate between them. The registration admitted that outcome in advance; answering the question needs experiments with substantial effects, and it stays open.

— Odds-Ratio Thompson Sampling: A Specification and Design Guide for Contrast-Based Multi-Armed Bandits  (2609.19709 - Kim, 17 Sep 2026) in Supplement E, “The registered secondary comparison: which scale is stable?”