Solvability of Atari games under the ATARI100k data budget

Determine whether all Atari games, including hard-exploration games, can be solved from scratch using the small data budget imposed by the ATARI100k benchmark.

Background

The ATARI100k benchmark evaluates reinforcement-learning agents after 100,000 agent steps, corresponding to 400,000 environment frames. The paper notes that this stringent budget may favor games with relatively easy exploration and may preclude meaningful learning in hard-exploration environments such as Pitfall! and Montezuma’s Revenge.

The unresolved issue is whether the benchmark’s limited data budget is sufficient, in principle or practice, for agents to solve the full range of Atari games from scratch rather than only a subset with favorable exploration characteristics.

References

While we consider this to be an important line of research, this tight budget produces an incentive to focus on a subset of games where exploration is easier, and it is unclear some games can be solved from scratch with such a small data budget.

— Human-level Atari 200x faster  (2209.07550 - Kapturowski et al., 2022) in Section 2, Related work, paragraph discussing the ATARI100k benchmark