Sample accounting and value of multi-agent self-play samples

Ascertain whether multi-agent samples collected by controlling all agents via self-play should be valued equivalently to per-time-step samples for measuring and comparing sample efficiency in reinforcement learning for driving simulators.

Background

The paper reviews concurrent simulators and training setups that control all vehicles via self-play, causing reported sample counts to scale with the number of agents. The authors point out the ambiguity in how such multi-agent samples should be counted and compared to standard time-step samples.

They explicitly state that the equivalence of these samples is unclear, suggesting a need for principled guidelines or empirical studies to standardize sample accounting and comparison.