VCNet: Bio-Inspired Visual Cortex Network

• VCNet is a bio-inspired neural architecture that replicates primate visual cortex organization with hierarchical processing, dual-stream segregation, and top-down predictive feedback.
• It leverages specialized modules such as depthwise separable convolutions, recurrent blocks, and attention mechanisms to mirror cortical dynamics and neural encoding.
• Empirical evaluations demonstrate robust performance in object recognition and neural response prediction, achieving up to 92.1% accuracy on select benchmarks.

Visual Cortex Network (VCNet) denotes, in its strictest usage, a bio-inspired neural architecture whose design is informed by the macro-scale organization of the primate visual cortex and that recreates hierarchical processing across distinct cortical areas, dual-stream information segregation, and top-down predictive feedback for artificial vision (Hill et al., 5 Aug 2025). In adjacent literature, the same label or a VCNet-style interpretation is also applied to self-organising visual cortex models, neural encoding systems that predict fMRI or MEG responses, representational analyses of cortex-wide visual networks, and visual-cortex-inspired deep architectures for recognition or temporal prediction (Luttrell, 2010, Meijer et al., 2019, Gifford et al., 2024, Canziani et al., 2017). Across these usages, the common theme is that vision is treated as a distributed cortical computation rather than a uniform feedforward stack.

1. Terminological scope and historical lineage

The literature summarized under VCNet is not terminologically uniform. One line uses the term explicitly: "VCNet: Recreating High-Level Visual Cortex Principles for Robust Artificial Vision" presents Visual Cortex Network as a directed acyclic graph that models the known connectivity of major visual cortical areas and uses biologically motivated modules for recognition tasks (Hill et al., 5 Aug 2025). Another early line uses the near-synonymous term VICON, a "visual cortex network" derived from a Bayesian analysis of individual neural firing events and designed to explain ocular dominance stripes and orientation maps from two retinae (Luttrell, 2010).

Related names mark distinct but overlapping agendas. CortexNet is introduced as a generic network family for robust visual temporal representations, with feed-forward, lateral, and feedback pathways inspired by visual cortex (Canziani et al., 2017). CVSNet is described as a computer implementation for the central visual system of the brain, organized around five non-identical blocks and three independent pathways rather than a repeated generic block (Gao et al., 2023). VISION, short for "Visual Interface System for Imaging Output of Neural activity," is a multimodal encoding model that predicts voxel-wise fMRI responses to natural images (Liang et al., 2023). A broader architectural proposal, "A Vision Architecture," treats cortex as an overlay of learned, dynamically activated nets formed slowly by learning and selected rapidly during perception (Malsburg, 2014).

This suggests that VCNet is best understood as both a specific architecture name and a broader research program. In the narrower sense, it refers to explicit visual-cortex-inspired neural networks. In the broader sense, it refers to models and analytical frameworks that attempt to reproduce cortical hierarchy, cortical encoding, cortical dynamics, or cortex-wide representational organization.

2. Core architectural motifs

The explicit VCNet architecture is organized as a graph of major visual cortical areas with two interacting pathways. Its ventral stream proceeds roughly through V1, V2 submodules including interstripe and thin stripe roles, V4, and PIT/CIT/AIT-like higher temporal modules; its dorsal stream proceeds through V1, V2 thick stripe, MT, MST, and higher parietal regions. The V1 module uses three parallel depthwise separable convolution streams with kernel sizes $3 \times 3$, $5 \times 5$, and $7 \times 7$. MT and MST use recurrent blocks with shared weights for a fixed number of iterations $t=3$. Selected modules incorporate CBAM attention, a lateral interaction block, neuromodulatory gating, and a top-down predictive coding loop from AIT to V1 (Hill et al., 5 Aug 2025).

Dual-stream segregation is a recurring motif well beyond the explicit VCNet paper. A dual-stream neural network explaining dorsal and ventral functional segregation uses WhereCNN and WhatCNN branches, retinal transformation on a $64 \times 64$ sample grid centered on fixation, and distinct foveation parameters $a=2.5$ for WhereCNN and $a=15$ for WhatCNN. WhereCNN learns spatial attention and outputs a saliency probability map with inhibition of return, while WhatCNN performs object recognition and integrates fixation-wise features with a GRU over eight fixations (Choi et al., 2023). The same feedforward-plus-recurrent logic appears in CortexNet, which organizes processing into paired discriminative and generative blocks $D_{1:L}$ and $G_{1:L}$, linked by bottom-up, top-down, and lateral connections (Canziani et al., 2017).

A contrasting architectural theme is the prioritization of early processing. The shallow residual neural network proposed for the 2019 Algonauts Human Brain Challenge is a ResNet-20, described as basically the same architecture as ResNet-18 but with the first ResNet block repeated 3 times instead of 2. The authors state that because they hypothesize the earliest layers are important, they add capacity there, arguing that earlier stages of the network can be accurately trained and that this permits more expressive early feature extraction for predicting visual cortex responses (Meijer et al., 2019).

Other VCNet-style systems push this cortical decomposition further. CVSNet is built from five different blocks—Inner Plexiform, Outer Plexiform, Lateral Geniculate Nucleus, Striate Cortex, and Abstract Cognitive Layer—and routes information through M, P, and K pathways, with the Striate Cortex block producing three pathways and six outputs (Gao et al., 2023). In "A Vision Architecture," the basic computational object is not a repeated block but a learned net embedded in permanent cortical connectivity, with winner-take-all selection and lateral support determining which net becomes active at perceptual time scales (Malsburg, 2014).

3. Neural encoding, representational alignment, and evaluation

VCNet research is evaluated through at least two distinct regimes: direct task performance on artificial benchmarks and alignment with biological responses. For neural-response prediction, the Algonauts setup uses 15 human subjects, ImageNet object images, two tracks—Track 1 fMRI and Track 2 MEG—and targets in EVC and IT. Responses are converted into representational dissimilarity matrices, predicted and recorded RDMs are compared using Spearman correlation, and the result is normalized against the correlation an ideal model could give (Meijer et al., 2019).

Representational similarity analysis is a central methodological bridge between cortical data and learned features. In the macaque IT study, ConvNet penultimate-layer activations are converted into class-level representational dissimilarity matrices, and similarity to IT dissimilarity structure is quantified by $s_{IT}$, the Spearman rank correlation between the upper-triangular, non-diagonal entries of model and IT RDMs, after adding noise to ConvNet activations to approximate measurement noise in neural recordings (Dubey et al., 2016). A later cortex-wide framework formalizes the same logic with

$5 \times 5$0

and combines inter-subject representational similarity, brain–model alignment across layers, and shared-component decomposition across NSD, BOLD5000, and THINGS-fMRI (Marcos-Manchón et al., 18 Jul 2025).

Voxelwise encoding models add a complementary evaluation regime. VISION predicts voxel-by-voxel fMRI activation across 27 anatomical regions from natural scenes, with an 80/20 train-test split per participant, and reports performance by noise-normalized accuracy, defined from voxelwise correlation relative to a noise ceiling (Liang et al., 2023). Relational neural control extends the evaluation target from single-area prediction to inter-area relationships: it generates controlling images that align or disentangle univariate and multivariate responses across V1, V2, V3, V4, EBA, FFA, PPA, and RSC, using RSA on representational similarity matrices and validating in silico discoveries on in vivo fMRI responses from independent subjects (Gifford et al., 2024).

4. Reported empirical performance

Reported results are heterogeneous because the literature spans object recognition, video modeling, voxel prediction, mouse V1 response prediction, and cortex-wide representational alignment. The figures below therefore describe task-specific outcomes rather than a single shared benchmark.

Model or study	Task	Reported result
VCNet (Hill et al., 5 Aug 2025)	Spots-10 animal pattern classification; light field image classification	$5 \times 5$1 on Spots-10; $5 \times 5$2 on the light field dataset
Shallow residual ResNet-20 (Meijer et al., 2019)	Visual cortex response prediction in Algonauts	Improvement from $5 \times 5$3 at block 1 to $5 \times 5$4 at the last fully connected layer
VISION (Liang et al., 2023)	Natural image to voxel-wise fMRI encoding	Accuracy exceeding state-of-the-art performance by $5 \times 5$5
Rotation-equivariant CNN (Ecker et al., 2018)	Mouse V1 neural response prediction	Test correlation $5 \times 5$6
CVSNet-2 (Gao et al., 2023)	ImageNet-1K / ILSVRC-2012 object recognition	$5 \times 5$7 top-1, $5 \times 5$8 top-5, $5 \times 5$9G params, $7 \times 7$0M FLOPs

Additional quantitative findings situate these results within broader cortex-model alignment. In the macaque IT comparison, representational similarity rises modestly with depth, from AlexNet $7 \times 7$1 to ResNet-152 $7 \times 7$2, while DeCov regularization increases IT similarity without necessarily increasing task accuracy and L1 regularization substantially reduces both similarity and accuracy (Dubey et al., 2016). At the cortex-wide level, parcel-wise vision-model alignment scales tightly with intersubject alignment via a power-law relationship, with $7 \times 7$3 in NSD, and the main pattern is reproduced in BOLD5000 and THINGS-fMRI with $7 \times 7$4 and $7 \times 7$5, respectively (Marcos-Manchón et al., 18 Jul 2025).

5. Biological interpretation and explanatory claims

A recurrent question in VCNet research is whether biologically aligned representations require greater depth, stronger recurrence, better early-layer training, or some combination of all three. The macaque IT study finds that deeper and better-optimized ConvNets are generally more IT-like, and that decorrelating features with DeCov can improve similarity even when validation accuracy does not improve (Dubey et al., 2016). By contrast, the Algonauts ResNet-20 paper argues that a shallow residual model may better capture visual cortex responses because earlier stages can be accurately trained and because additional early capacity improves prediction from $7 \times 7$6 to $7 \times 7$7 (Meijer et al., 2019). This suggests that the relation between depth and cortical alignment is task-dependent, species-dependent, and level-dependent rather than monotonic in a universal sense.

At the level of local cortical computation, several studies revise textbook simplifications. The rotation-equivariant CNN for mouse V1 learns a shared orientation-invariant feature space and reveals common features that deviate from the typical idea of V1 as a bank of Gabor filters, including center-surround structure and asymmetric, crescent-shaped surrounds (Ecker et al., 2018). Rapid contextual learning in a ViT-based autoencoder argues that familiarity training aligns early layers with the top layer that contains global context information, broadens self-attention scope within the remembered image context, and that these effects are significantly amplified by LoRA-based fast weights (Li et al., 7 Aug 2025). A plausible implication is that early visual cortex is being modeled increasingly as a dynamically context-sensitive system rather than a purely local filter bank.

At the network level, the emphasis shifts from isolated areas to shared geometry and pathway specialization. The convergent-transformations study identifies a cortex-wide network organized into a medial-ventral stream for scene structure and a lateral-dorsal stream tuned for social and biological content, and reports that this organization is captured by the hierarchies of vision DNNs but not LLMs (Marcos-Manchón et al., 18 Jul 2025). Relational neural control complements this by showing that shared and unique representational content varies with cortical distance, categorical selectivity, and hierarchical position: in V1–V4 comparisons, unique univariate content emphasizes high spatial frequency in V1 and object-like shapes in V4, while shared multivariate content is explained by retinotopic structure (Gifford et al., 2024). In older self-organising formulations such as VICON, the same network-level ambition appears in developmental terms: local probabilistic coding and reconstruction pressure are sufficient for ocular dominance stripes and orientation maps to emerge without explicitly hard-coding eye labels or orientation structure (Luttrell, 2010).

6. Limitations, debates, and research directions

The literature does not present a single canonical VCNet definition. Some works use the term formally for a classifier architecture, others use a VCNet-style interpretation for encoding models or representational analyses, and others adopt adjacent names such as VICON, CortexNet, CVSNet, or VISION. This terminological breadth is scientifically productive, but it also means that empirical comparisons are often cross-task, cross-species, and cross-objective rather than like-for-like.

Several limitations are explicit in the underlying papers. The explicit VCNet architecture is described as ambitious but still early-stage: it has no extensive ablation study, limited benchmark variety, no adversarial robustness or out-of-distribution tests, a simplified predictive coding mechanism, and only approximate biological realism at the macro-architectural level (Hill et al., 5 Aug 2025). The macaque IT similarity study is described as preliminary, compares only the penultimate layer, uses a specific macaque IT dataset, reports modest effects overall, and does not establish causality between recognition performance and cortical similarity (Dubey et al., 2016). The cortex-wide convergence study emphasizes that convergence is not the absence of variability and that THINGS-fMRI yields a more restricted pattern largely limited to early visual cortex, consistent with its simpler object-centric stimuli and orthogonal oddball task (Marcos-Manchón et al., 18 Jul 2025).

Even with these limitations, the direction of travel is clear. VCNet research increasingly combines macro-scale cortical organization, voxelwise or population-level neural prediction, representational geometry, recurrence, contextual plasticity, and interpretable probes. VISION explicitly argues that, with both a model and an evaluation metric, the cost and time burdens associated with designing and implementing functional analysis on the visual cortex could be reduced (Liang et al., 2023). A plausible implication is that future VCNet work will be judged not only by recognition accuracy, but also by how well it unifies robust artificial vision, cortical alignment, and experimentally testable accounts of visual computation.