Patch-level credit assignment for sparse visual selection

Investigate improved patch-level credit-assignment methods for reinforcement-learning-based sparse visual selection, with the aim of overcoming the difficulty of assigning rewards to individual patches when rewards are currently assigned to the full selection mask.

Background

LookThere trains a selector through reinforcement learning to choose a sparse mask of high-resolution image patches, while an extractor evaluates the selected content through the downstream task loss. Because the reward is assigned to the complete selection mask rather than to individual patches, the learning signal does not directly identify which patches were responsible for successful or unsuccessful predictions.

The paper states that grouped comparisons in Group Relative Policy Optimization (GRPO) partly mitigate this limitation, but leaves the development of more effective patch-level credit-assignment mechanisms unresolved. Improving this aspect could make reinforcement-learning-based sparse visual selection more reliable and better optimized.