Attention when you need (2501.07440v2)

Abstract: Being attentive to task-relevant features can improve task performance, but paying attention comes with its own metabolic cost. Therefore, strategic allocation of attention is crucial in performing the task efficiently. This work aims to understand this strategy. Recently, de Gee et al. conducted experiments involving mice performing an auditory sustained attention-value task. This task required the mice to exert attention to identify whether a high-order acoustic feature was present amid the noise. By varying the trial duration and reward magnitude, the task allows us to investigate how an agent should strategically deploy their attention to maximize their benefits and minimize their costs. In our work, we develop a reinforcement learning-based normative model of the mice to understand how it balances attention cost against its benefits. The model is such that at each moment the mice can choose between two levels of attention and decide when to take costly actions that could obtain rewards. Our model suggests that efficient use of attentional resources involves alternating blocks of high attention with blocks of low attention. In the extreme case where the agent disregards sensory input during low attention states, we see that high attention is used rhythmically. Our model provides evidence about how one should deploy attention as a function of task utility, signal statistics, and how attention affects sensory evidence.

• The paper introduces a reinforcement learning-based normative model that quantifies the trade-off between attention costs and reward-driven task performance.
• The model shows that agents strategically alternate between high and low attention, closely emulating experimental behavior under varying reward conditions.
• The study reveals that rhythmic patterns in attention deployment emerge when disengagement yields no sensory gain, offering novel predictions about neural dynamics.

The paper "Attention when you need" explores strategic allocation of attention in tasks where attention has both benefits in improving task performance and costs associated with its deployment. Using a reinforcement learning-based normative model, the work investigates this balance by simulating a task involving mice, inspired by experiments conducted by de Gee et al. The task requires discerning a high-order acoustic feature amidst noise, with attention costs varying with the trial duration and magnitude of reward.

Key components of the model include:

• States and Actions: The latent world states transition through noise and signal phases, and the agent (the simulated mouse) can choose whether to lick (indicating decision) and whether to pay low or high attention at each discrete time step. This decision influences the subsequent observations.
• Observations and Rewards: Binary observations with stochastic noise are provided, where high attention incurs metabolic costs but yields more reliable observations. The agent receives rewards based on the timing of licks during the signal phase, and false alarms (licks during noise) result in penalties.

The paper shows that the agent allocates attention economically, shifting between blocks of high and low attention. When presented with the option to disengage completely during certain phases (allowing zero information gain, $p_{\rm low} = 0.5$), rhythmic attention patterns emerge. This suggests that agents devise strategies to optimize task utility, which aligns with observations of attention rhythms in neuroscience literature.

Important results include:

1. Behavioral Emulation: The trained agent displays behavioral trends akin to experimental observations—higher food rewards lead to increased hit rates, false alarms, and reduced reaction times.
2. Attention Dynamics: The agent's decision to deploy high attention is sensitive to its belief in the presence of the signal and the magnitude of the reward. Increased rewards and signal certainty encourage higher attention.
3. Rhythmic Patterns: When disengagement provides no sensory information, high attention instances follow rhythmic patterns, with wait times between blocks of high attention being regular. This temporal structure is driven by cost constraints and suggests optimization principles in attention deployment.

The paper suggests that periodic attention patterns previously observed in vision tasks may be normatively justified by the costs of attention. The results also propose empirical predictions regarding neural and physiological correlates of attention, advocating that fluctuations seen in neural signals might correspond to the fluctuating strength of evidence as attention varies. These findings may provide insights into understanding attention lapses and rhythmic attention deployment in animal and human behaviors.

The work underscores the need for future research to refine models with continuous observation spaces, incorporate switching costs between attention states, and further explore how attention policies evolve in different contexts. The model could have broader implications for understanding cognitive processes across various modalities and conditions where attention is a costly resource.