Mechanism of temporal information compression and integration in recurrent agents

Characterize how the recurrent neural network agents trained on the ViZDoom foraging task compress past sensory information and integrate it with incoming visual observations to support survival, especially in the CIFAR-10 condition where temporal integration was critical.

Background

The paper finds that recurrent architectures are crucial to fully exploit complex vision models on visually demanding tasks, notably when object classes are represented by CIFAR‑10 images. This implies a temporal integration mechanism that compresses and accumulates information over time.

The authors explicitly list as an open question how their recurrent agents perform this compression and integration of past and current sensory information, noting its critical role in the most complex task. They suggest this process is analogous to Kalman filtering, motivating a deeper mechanistic analysis.