Vision: looking and seeing through our brain's information bottleneck (2503.18804v1)

Abstract: Our brain recognizes only a tiny fraction of sensory input, due to an information processing bottleneck. This blinds us to most visual inputs. Since we are blind to this blindness, only a recent framework highlights this bottleneck by formulating vision as mainly looking and seeing. Looking selects a tiny fraction of visual information for progression through the bottleneck, mainly by shifting gaze to center an attentional spotlight. Seeing decodes, i.e., recognizes, objects within the selected information. Since looking often occurs before seeing and evokes limited awareness, humans have the impression of seeing whole scenes clearly. According to the new framework, the bottleneck starts from the output of the primary visual cortex (V1) to downstream brain areas. This is motivated by the evidence-backed V1 Saliency Hypothesis (V1SH) that V1 creates a saliency map of the visual field to guide looking. Massive visual information loss downstream from V1 makes seeing vulnerable to ambiguity and illusions (errors). To overcome this, feedback from downstream to upstream areas such as V1 queries for additional relevant information. An integral part of this framework is the central-peripheral dichotomy (CPD) theory proposing that vision in the peripheral and central visual fields are specialized for looking (deciding where to shift the gaze) and seeing, respectively, and that the feedback query to aid seeing is mainly directed to the central visual field. This V1SH-Bottleneck-CPD framework predicts that the peripheral visual field, lacking feedback queries, is more vulnerable to illusions, and that such illusions become visible in the central visual field when the feedback query is compromised. We present theoretical predictions, experimental confirmations, a Feedforward-Feedback-Verify-and-reWeight (FFVW) algorithm for seeing through the bottleneck.

Vision: Through the Brain's Information Bottleneck

Li Zhaoping's paper provides a novel framework for understanding visual perception by considering the brain's information processing bottleneck. The paper argues for a theoretical framework that acknowledges the limitations in sensory data processing by our brains, revisiting key concepts such as selective attention and perceptual inference through the V1 Saliency Hypothesis (V1SH), the Bottleneck, and the Central-Peripheral Dichotomy (CPD) theory.

At the center of this discussion lies the bottleneck that begins at the primary visual cortex (V1), which significantly reduces the vast amount of information received from the retina down to what is manageable for further cognitive processing. The framework differentiates between "looking" and "seeing" as two principal activities of vision: "looking" involves selecting a small subset of visual information to pass through the processing bottleneck, while "seeing" involves the recognition of objects within this selected information.

The paper presents compelling evidence that V1 creates a saliency map to guide gaze direction, an assumption supported by various studies indicating V1's role in determining the "salient" aspects of a visual scene. This framework also aligns with the V1SH, suggesting that visual illusion vulnerabilities upstream are mitigated by feedback processes downstream—primarily in central vision—where additional relevant sensory information is queried to assist in seeing.

Several novel predictions arise from this framework, some experimentally confirmed. The paper outlines a potential algorithm, termed Feedforward-Feedback-Verify-and-reWeight (FFVW), that operates within this framework, predicting not only the tasks of resolving perceptual ambiguities and visual illusions but also how these processes are specialized in central versus peripheral vision. The FFVW approach embodies analysis-by-synthesis, using feedback from downstream to verify and influence upstream processes, primarily in central vision. Peripheral vision, lacking such feedback mechanisms, tends to be more susceptible to illusions and is specialized for broader visual orienting.

Two key phenomena where this framework has been substantiated experimentally include the flip tilt illusion and the reversed depth illusion, primarily observed in peripheral vision due to the absence of feedback queries. In the case of the reversed depth illusion, where traditionally only central vision experiments were considered, the CPD theory-led research indicates that these illusions are indeed perceivable in the periphery, advocating for broader experimental paradigms.

The work critiques earlier theories of vision lacking precise formulations and not accounting for the bottleneck's presence. By emphasizing the importance of the bottleneck as both a theoretical and practical limiting factor that scales all cognitive interpretations of vision, the paper bridges perceptual psychology concepts with neurophysiology. The attention drawn to seemingly simple processes uncovers complex interactions between various regions of the brain.

From a theoretical standpoint, the paper posits that significant information loss occurs after V1, urging for investigations into gradual information loss across different stages of the visual pathway. This invites a re-examination of neuroanatomical data about the visual system, with potential implications for designing neural networks in artificial intelligence, as they might adopt and simulate the spare selection process dictated by human's natural bottleneck mechanisms.

Overall, the paper presents a comprehensive and integrated view of vision that may shape future psychological and cognitive neuroscience research strategies. By exploring the specificity of these visual processes and illustrating them with empirical evidence, Zhaoping sets a formal stage for assessing the neurophysiological bases of visual perception, orienting research towards understanding these complex interactions in a measurable and experimentally verifiable manner. This work challenges the field to further explore both the constraints and capabilities imposed by the brain’s inherent processing limitations. Future research could expand this framework, examining how it applies to various sensory modalities and incorporating potential interactions between the visual system and other cognitive processes.