Modeling Adaptive Visual Search in Semantically Hierarchical Layouts

Abstract: This paper introduces a computational cognitive model to investigate how information grouping impacts visual search, a key consideration in user interface design. The model uses computational rationality to view user behavior as an adaptation to cognitive and task constraints. Our work highlights that humans use hierarchical task representations, exploiting semantic and visual structures to improve search efficiency within the constraints of the visual system. We validate this model with data from two human studies focused on visual search and semantic categorization, demonstrating that semantic grouping improves search performance when it aligns with spatial grouping. Our model replicates task durations and eye movement patterns. By improving understanding of how hierarchical memory structures are utilized in human cognition, the model extends previous visual search models. We showcase our model in the rapid prototyping and evaluation of semantic visual groupings within user interface wireframes, suggesting a pathway toward applications in more complex, real-world interface design.