Mechanism and robustness of the K=8 dispersion advantage

Investigate why the K-Means-partitioned Mixture-of-Experts architecture with K=8 experts exhibits lower across-seed dispersion than vanilla Double Deep Q-Learning, and determine whether the apparent advantage is genuine or results from selection on the tested three-point K grid.

Background

The experiments find that the K=8 Mixture-of-Experts configuration has the lowest across-seed dispersion among the tested learned configurations. However, the effect is non-monotone in the number of experts, does not survive family-wise adjustment, is not explained by the removal of collapsed runs, and narrows under the modified training specification. The authors therefore do not identify a mechanism that accounts for the observed dispersion pattern and caution that the K=8 result may reflect selection from testing only K=2, 4, and 8.

References

The lower dispersion at $K{=}8$ is therefore mechanistically unresolved, and since $K{=}4$ is the weakest MoE configuration on both mean and dispersion, a benefit appearing only at $K{=}8$ may reflect selection across a three-point grid.

— Mixture-of-Experts for Cryptocurrency Order Execution: Training Stability, Tail Risk, and Failure Modes  (2610.03369 - Ardaiz et al., 2 Oct 2026) in Section 4, subsection “Baseline reference and degenerate runs”