Recurrent computations for visual pattern completion (1706.02240v2)

Abstract: Making inferences from partial information constitutes a critical aspect of cognition. During visual perception, pattern completion enables recognition of poorly visible or occluded objects. We combined psychophysics, physiology and computational models to test the hypothesis that pattern completion is implemented by recurrent computations and present three pieces of evidence that are consistent with this hypothesis. First, subjects robustly recognized objects even when rendered <15% visible, but recognition was largely impaired when processing was interrupted by backward masking. Second, invasive physiological responses along the human ventral cortex exhibited visually selective responses to partially visible objects that were delayed compared to whole objects, suggesting the need for additional computations. These physiological delays were correlated with the effects of backward masking. Third, state-of-the-art feed-forward computational architectures were not robust to partial visibility. However, recognition performance was recovered when the model was augmented with attractor-based recurrent connectivity. These results provide a strong argument of plausibility for the role of recurrent computations in making visual inferences from partial information.

• The paper provides compelling evidence from behavioral experiments, neurophysiological recordings, and computational modeling that recurrent computations are critical for visual pattern completion.
• Humans are remarkably robust at recognizing highly occluded objects, but backward masking disrupts this, suggesting temporal processing involving recurrent loops.
• Standard feed-forward models fail on partial objects, but adding recurrent connections enhances performance to match human ability, supporting the computational role of recurrence.

Analysis of Recurrent Computations for Visual Pattern Completion

The paper "Recurrent computations for visual pattern completion" by Tang et al. examines the intrinsic processes involved in making inferences from incomplete visual information, a fundamental aspect of cognitive perception. The paper proposes that pattern completion in visual perception involves recurrent computations and provides experimental evidence supporting this hypothesis.

Experimental Approach

The authors employ a comprehensive approach integrating psychophysics, neurophysiology, and computational modeling to assess recognition of partially visible objects. They utilize behavioral experiments where subjects recognize objects with varying levels of visibility, supplemented by neurophysiological data from recordings in the human ventral visual cortex. Additionally, the paper involves testing computational models against these empirical observations.

Findings

Three primary findings bolster the hypothesis of recurrent computations:

1. Behavioral Robustness to Occlusion: The paper finds that human subjects are adept at recognizing highly occluded objects, maintaining high performance levels even at less than 15% visibility. However, introducing backward masking severely disrupts this recognition ability, suggesting that unmasked recognition benefits from extended temporal computations.
2. Neurophysiological Delays: The research documents delayed selective neural responses to partially visible objects, as opposed to whole objects, along the human ventral cortex. These delays correlate with the behavioral disruptions observed during backward masking, implying a reliance on additional computations for pattern completion.
3. Inadequacy of Feed-Forward Models: Initial assessments reveal that state-of-the-art feed-forward computational models, such as AlexNet and others, do not exhibit robustness to partial visibility. However, incorporating attractor dynamics via recurrent connections notably enhances model performance, offering achieved parity with human-like behavior under these conditions.

Implications on Computational Models

The augmentation of feed-forward models with recurrent connections proposes a viable mechanism underlying human-like robust recognition in computational systems. By simulating an attractor network, the additions help models better emulate the temporal integration seen in human pattern completion. Specifically, the Hopfield network-like dynamics allow the models to iteratively refine representations of partial objects, drawing them closer to whole object representations, as observed through improved classification accuracy and correspondence with human data.

Theoretical and Practical Implications

Theoretically, this paper reinforces the notion of temporal and spatial integration as essential mechanisms in the brain's pattern completion processes, with recurrent neural circuits playing a pivotal role. The backward masking experiments support the idea that such recurrent processes inhabit extended windows of cognitive processing.

Practically, these findings could influence the design of computer vision systems tasked with interpreting occluded or partially visible data. Recurrent network architectures could provide enhanced capabilities in environments with visual obstructions, benefiting fields such as autonomous navigation and robotics.

Speculation on Future Research

Further investigation could explore the incorporation of additional sensory and contextual information in recurrent models, simulating multi-modal integration in human perception. Another promising avenue involves extending these models to include variable, flexible layers of recurrence, more closely mimicking human adaptability in response to different visual environments.

In conclusion, through a confluence of behavioral, physiological, and computational paradigms, Tang et al. provide compelling evidence for the critical function of recurrent computations in human visual pattern completion. This work stimulates both theoretical discourse and practical advancements in our understanding and emulation of complex cognitive processes.