Usefulness of the Nibbler approach on non-synthetic (real-world) data

Determine whether the Nibbler algorithm and its GVF-based feature construction approach are useful for non-synthetic (real-world) data streams with unstructured observations.

Background

The paper evaluates Nibbler on synthetic multi-catch environments specifically designed to test scaling with unstructured observations. The authors highlight that these experiments do not establish applicability to real-world data.

They state that further work is required to assess whether the approach provides practical benefits on non-synthetic data, explicitly identifying this as an unresolved question.

References

Further work is required to see whether this approach is useful for non-synthetic data (where data comes from the real world).

— Towards model-free RL algorithms that scale well with unstructured data (2311.02215 - Modayil et al., 2023) in Section 7 (Discussion)