Foveated Retinotopy Improves Classification and Localization in CNNs

Abstract: From a falcon detecting prey to humans recognizing faces, many species exhibit extraordinary abilities in rapid visual localization and classification. These are made possible by a specialized retinal region called the fovea, which provides high acuity at the center of vision while maintaining lower resolution in the periphery. This distinctive spatial organization, preserved along the early visual pathway through retinotopic mapping, is fundamental to biological vision, yet remains largely unexplored in machine learning. Our study investigates how incorporating foveated retinotopy may benefit deep convolutional neural networks (CNNs) in image classification tasks. By implementing a foveated retinotopic transformation in the input layer of standard ResNet models and re-training them, we maintain comparable classification accuracy while enhancing the network's robustness to scale and rotational perturbations. Although this architectural modification introduces increased sensitivity to fixation point shifts, we demonstrate how this apparent limitation becomes advantageous: variations in classification probabilities across different gaze positions serve as effective indicators for object localization. Our findings suggest that foveated retinotopic mapping encodes implicit knowledge about visual object geometry, offering an efficient solution to the visual search problem - a capability crucial for many living species.

• The paper demonstrates that integrating retinotopic mapping into CNNs improves robustness against geometric distortions such as rotations and zooms.
• It leverages a logarithmic-polar transform to simulate foveated vision, converting complex image transformations into simpler translations.
• Empirical tests on Vgg16 and ResNet101 reveal that the approach yields comparable or improved classification and localization performance.

Enhancing Convolutional Neural Networks with Retinotopic Mapping

Introduction to Retinotopic Mapping in CNNs

In an exploration of improving image classification and localization tasks, a novel approach integrating retinotopic mapping into Convolutional Neural Networks (CNNs) is presented. This study focused on adjusting CNNs' input phase with a retinotopic transform—a feature inspired by the foveated vision common in many animal species. The methodology involved retraining widely recognized CNN architectures with images transformed via logarithmic-polar mapping, aiming to examine the impact on the networks' ability to handle image rotations and scale changes.

Core Findings and Experimental Insights

Robustness to Geometric Perturbations

A striking outcome of the study is the demonstration that CNNs, when adjusted with retinotopic mapping, exhibit enhanced robustness against geometric transformations of input images, specifically zooms and rotations. This finding contrasts with the common vulnerability of traditional CNNs to adversarial attacks, highlighting the potential of retinotopic mapping in crafting more resilient image recognition systems. These vanilla CNNs were shown to significantly falter under rotation-based attacks, underlining the importance of considering realistic perturbations in model evaluations.

Advantages of Biologically Inspired Mapping

The research underpins the computational benefits of biologically inspired visual processing models. By mimicking the human visual system's retinotopic organization, where a logarithmic scale dictates the distribution of photoreceptors in the retina, the transformed input to CNNs showcases an inherent invariance to scale and rotation. This is attributed to the property of the log-polar transform, which converts zooms and rotations in the visual space into simpler translations in the transformed space, thereby leveraging the convolutional layers' translation invariance in a novel manner.

Empirical Validation and Theoretical Implications

The experimental validation involved retraining the Vgg16 and ResNet101 networks on the ImageNet dataset with the retinotopic transformation engaged. Remarkably, these retinotopically mapped networks achieved comparable, and in some dimensions improved, performance against their traditionally trained counterparts. This result is non-trivial, as it suggests that despite significant distortion and information compression inherent in the retinotopic mapping, networks can still extract and utilize relevant features for accurate classification. Additionally, the study delved into object localization capabilities, finding improved localization performance when analyzing images with varying focal points. This mirrors a critical aspect of human visual attention not previously encapsulated by standard CNN designs.

Future Directions in AI and Robust Vision Systems

The integration of retinotopic mapping opens up new avenues for research in geometric deep learning and AI vision systems. It suggests a pathway towards developing models that not only match but potentially surpass human visual recognition capabilities under varied and challenging conditions. The study's insights into the naturally robust preattentive processing mechanisms present an exciting frontier for improving AI systems' efficiency and resilience.

Moreover, this approach could inspire future models that incorporate dynamic foveation mechanisms, mimicking saccadic eye movements to iteratively focus on regions of interest within an image. Such models could vastly improve the computational efficiency and accuracy of tasks like visual search, further bridging the gap between artificial and biological vision systems.

In conclusion, the exploration of retinotopic mapping within CNNs marks a significant step towards harnessing biologically inspired principles to bolster AI robustness and capability. It not only demonstrates the practical advantages of such an approach in current architectures but also sets the stage for profound advancements in how machines perceive and understand the visual world.