Interactions involving replay-buffer retention

Determine how replay-buffer retention interacts with reward alignment and exploration annealing in vanilla Double Deep Q-Learning, the K=8 Mixture-of-Experts architecture, and the capacity-matched dense network, completing the untested two-change configurations.

Background

The specification decomposition examines reward alignment, exploration annealing, and replay-buffer retention, but does not run two of the three possible two-change configurations involving replay retention. Although changing replay retention alone produces no detectable effect on collapse rates, the available experiments do not determine whether replay retention interacts with reward alignment, exploration annealing, or both. The authors explicitly identify these interactions as unresolved.

References

Changing replay retention alone has no detectable effect on collapse rates, so the reward-plus-annealing arm establishes a reward-by-exploration interaction; interactions involving replay retention are unresolved, two of the three two-change arms not having been run.

— Mixture-of-Experts for Cryptocurrency Order Execution: Training Stability, Tail Risk, and Failure Modes  (2610.03369 - Ardaiz et al., 2 Oct 2026) in Section 4, subsection “Specification Decomposition”