MLP-Based Visual Models

Updated 11 July 2026

MLP-based visual models are fully connected systems that process image patches by alternating token and channel mixing instead of using spatial convolutions.
They integrate innovations like axial projections, spatial shifts, dynamic mixing, and gating to overcome limitations such as overfitting and resolution sensitivity.
They are applied across diverse domains including image classification, restoration, 3D recognition, and multimodal fusion, often as part of hybrid architectures.

Searching arXiv for the papers on arXiv and closely related MLP-based visual model references. Multi-layer perceptron-based visual models comprise a family of vision systems in which the principal computation is performed by fully connected projections rather than by spatial convolution or self-attention. In their modern form, these models typically operate on image patches or structured tensors and alternate spatial or token mixing with channel mixing, although important subclasses instead use axial projections, shifts, gating, dynamic mixing, orthogonal transforms, or multi-output depth supervision. The literature therefore spans early classifiers over handcrafted image descriptors, all-MLP ImageNet backbones, low-level restoration networks, multi-view 3D recognizers, multimodal remote-sensing classifiers, and coordinate-based signal representations (Liu et al., 2021, Wang et al., 2023, Yang et al., 26 Aug 2025).

1. Historical formation and conceptual scope

The history of MLP-based visual modeling is not confined to the recent all-MLP revival. Earlier systems used MLPs as discriminative classifiers over engineered image representations. In handwritten Arabic numeral recognition, a feature set of 88 features, including 72 shadow and 16 octant features, was coupled to a single-hidden-layer MLP and yielded an average recognition rate of 94.93% on a database of 3000 samples after three-fold cross validation (Das et al., 2010). In visual face recognition, a later pipeline combined log-polar transformation, PCA/eigenfaces, and an MLP classifier to address rotation and scale variability (Bhowmik et al., 2010).

The modern resurgence redefined the MLP as the backbone itself rather than merely the final classifier. The survey literature formalized this shift by treating convolution, self-attention, and token-mixing MLPs as different weighted-sum paradigms: convolution as static local aggregation, self-attention as dynamic global aggregation, and token-mixing MLP as static global aggregation (Liu et al., 2021). In parallel, ResMLP presented a pure feed-forward residual network for image classification that used no attention, no positional embeddings, no class token, and no normalization based on statistics, while remaining competitive under modern training (Touvron et al., 2021).

This broader history matters because it shows that “MLP-based visual models” is not a single architectural template. It includes at least three regimes: feature-engineered visual classifiers with MLP heads, all-MLP recognition backbones, and MLP-centric hybrids or regressors designed for restoration, multimodal inference, or continuous signal representation. This suggests that the term designates a design philosophy centered on learned affine mixing, rather than a uniquely defined network family.

2. Fundamental computational patterns

At the core of the contemporary family is the separation between spatial or token mixing and channel mixing. In the survey taxonomy, an image is first partitioned into patches, embedded as tokens, and then processed by alternating MLPs along the token axis and the channel axis; channel mixing is functionally equivalent to a $1\times1$ convolution, while token mixing implements a dense learned projection over spatial positions (Liu et al., 2021). This decomposition yielded the canonical MLP-Mixer-style block, but it also exposed the central problem of the field: global token mixing is expressive yet resolution-sensitive and computationally expensive.

One response was to preserve the two-dimensional structure instead of flattening it. Vision Permutator separately encodes height and width with linear projections, thereby maintaining positional precision along one axis while mixing globally along the other. Its Permute-MLP fuses height, width, and channel branches, optionally with split-attention weighting, and thereby turns axis-wise projections into a substitute for flattened token mixing (Hou et al., 2021). X-MLP pushed this logic further by removing patch embedding entirely and decoupling width, height, and channel interactions directly on the image tensor (Wang et al., 2023).

A second response was to replace learned global token projections with spatially structured data movement. S $^2$ -MLP discards token-mixing MLP altogether and uses a parameter-free spatial-shift operation that divides channels into groups and shifts them in cardinal directions before channel mixing. The stated motivation is that token-mixing MLP behaves like a depthwise operator with a global receptive field and spatial-specific configuration, which is prone to over-fitting on medium-scale datasets; the shift operator instead imposes locality and spatial agnosticism (Yu et al., 2021). Hire-MLP generalizes this idea through inner-region rearrangement for local modeling and cross-region rearrangement for global communication, while R $^2$ -MLP extends spatial shift to the view dimension by rolling channel groups across neighboring views in a circular topology (Guo et al., 2021, Chen et al., 2022).

A third response was to retain global mixing but make it adaptive or gated. DynaMixer generates a content-conditioned mixing matrix $P \in \mathbb{R}^{N\times N}$ from the tokens themselves, computes $Y = P X$ , and reduces generator cost by dimensionality reduction and multi-segment fusion (Wang et al., 2022). MAXIM, designed for image processing rather than recognition, uses a multi-axis gated MLP that mixes local blocked windows and global dilated grids, plus a cross-gating block that conditions one feature stream on another without attention (Tu et al., 2022). In a different direction, MC-MLP applies fixed orthogonal transforms of the form $Y = Q_N X Q_C^\top$ , using DCT and Hadamard transforms to expose the same content in multiple coordinate frames before MLP processing (Zhu et al., 2023).

These operator families differ in what they treat as the central deficiency of vanilla token mixing. Axis factorization addresses loss of two-dimensional structure, shift and rearrangement address overfitting and resolution rigidity, dynamic mixing addresses content agnosticism, and orthogonal-frame methods address the possibility that the same semantics are easier to learn in transformed bases. A plausible implication is that much of the field’s innovation has consisted of reintroducing selective inductive bias into a nominally bias-light architecture class.

3. Architectural families and representative systems

The major architectural lineages can be organized by how they mix spatial information, whether they preserve hierarchy, and how much operator purity they retain.

Model	Structural idea	Reported result
ResMLP (Touvron et al., 2021)	Single-stage residual stack with global token linear mixing and per-token channel MLP	81.0% top-1 on ImageNet-1k for ResMLP-B24
Vision Permutator (Hou et al., 2021)	Height, width, and channel branches with axis-wise projections and split-attention fusion	83.2% top-1 on ImageNet-1k for ViP-Large/7
DynaMixer (Wang et al., 2022)	Dynamic, content-conditioned row/column mixing with multi-segment fusion	84.3% top-1 on ImageNet-1K for DynaMixer-L
MAXIM (Tu et al., 2022)	UNet-shaped multi-axis gated MLP with cross-gating for image processing	39.96 dB PSNR and 0.960 SSIM on SIDD
Hire-MLP (Guo et al., 2021)	Hierarchical rearrangement with inner-region and cross-region mixing	83.8% top-1 on ImageNet; 51.7 box AP and 44.8 mask AP on COCO; 49.9 mIoU on ADE20K
BiMLP (Xu et al., 2022)	Binary MLP with multi-branch blocks and universal shortcuts	72.7% Top-1 on ImageNet-1k at 0.188G OPs for BiMLP-M

Two broad structural divisions are especially important. The first is isotropic versus hierarchical design. Early all-MLP models such as ResMLP remained close to the ViT-like single-stage template, whereas ViP, Hire-MLP, ConvMLP, DynaMixer, and BiMLP adopted multi-stage or pyramidal structures to improve efficiency, multi-scale processing, and downstream transfer (Li et al., 2021). The second is pure versus hybrid operator choice. X-MLP and ResMLP aimed at purely fully connected processing, whereas ConvMLP explicitly coupled depthwise and pointwise convolutions with channel MLPs to avoid fixed-resolution spatial MLPs and to produce transferable feature pyramids (Wang et al., 2023, Li et al., 2021).

In practice, the most successful architectures frequently sacrifice strict purity. MAXIM is “exclusively based on MLPs” in its two principal mixing modules, yet it also incorporates residual channel attention blocks with convolutions (Tu et al., 2022). ConvMLP is overtly a co-design of convolution layers and MLPs (Li et al., 2021). BiMLP retains the first layer, last layer, and downsampling layers in full precision (Xu et al., 2022). This suggests that the architectural center of gravity moved from replacing every non-MLP component to identifying where MLP mixing is most advantageous and where auxiliary structure remains useful.

4. Task domains and empirical case studies

MLP-based visual models appear across markedly different task regimes, from small-data recognition to large-scale image classification and continuous signal regression.

In classical recognition, the MLP often acts as a final classifier after domain-aware preprocessing. The handwritten Arabic numeral system used handcrafted shape descriptors and a single-hidden-layer MLP over 88 features, obtaining 94.93% average recognition after three-fold cross validation (Das et al., 2010). The log-polar eigenface pipeline for face recognition used Cartesian-to-log-polar conversion to turn rotation into angular shifts and scaling into radial-log offsets, then projected the transformed images into a 40-dimensional eigenspace and classified them with a two-hidden-layer MLP. The reported recognition rates rose from 89.5% to 97.5% on ORL and from 87.84% to 96.36% on OTCBVS when moving from visual eigenfaces plus MLP to log-polar-visual eigenfaces plus MLP (Bhowmik et al., 2010).

In recommendation, visual inputs enter the MLP through multimodal fusion rather than direct classification. MF-VMLP extracts 4096-dimensional image features using a pre-trained CAFFE reference model, projects them through a linear embedding $\theta_i = E f_i$ , and fuses collaborative-filtering embeddings with visual factors in a NeuMF-style architecture. On Amazon Women, the reported RMSE values are 1.1303 for MF, 1.0886 for VMF, 1.0639 for VMLP, and 1.0575 for MF-VMLP; on Amazon Men they are 1.0579, 1.0287, 1.0193, and 1.0034, while Amazon Phones shows much smaller gains (Lin et al., 2019).

In remote sensing, SGU-MLP combines a multi-scale depth-wise convolutional front-end, Mixer-style token and channel MLPs, and spatial gating units for land use and land cover mapping. The reported overall accuracies are 87.91 on Houston, 70.79 on Berlin, and 91.82 on Augsburg, and the ablations show cumulative gains from adding SGU and DWC to the baseline mixer (Jamali et al., 2023). In multi-view 3D recognition, R $^2$ -MLP extends spatial-shift MLP to a cyclic view graph. With 20 views, it reports 97.7% on ModelNet40 and 99.6% on ModelNet10; with 12 views, it reports 95.0% and 97.4%, respectively (Chen et al., 2022).

In low-level vision, MAXIM demonstrates that MLP-based backbones are not restricted to classification. It reports 39.96 dB PSNR and 0.960 SSIM on SIDD, 32.86 dB and 0.961 SSIM on GoPro, 38.11 dB and 0.991 SSIM on RESIDE SOTS Indoor, and 26.15 dB and 0.945 SSIM on MIT-Adobe FiveK, while using a fully-convolutional UNet-shaped hierarchy with multi-axis gated MLPs (Tu et al., 2022). T-MLP extends the scope further into level-of-detail implicit representations by attaching output tails to hidden layers and supervising cumulative outputs across depths. In 3D SDF fitting, T-MLP reports Thingi10K Chamfer Distance mean 1.740 and Normal Consistency mean 99.39, with 266k parameters; in image fitting it reports 37.60 PSNR and 96.82 SSIM at $512\times512$ , and 30.63 PSNR and 88.52 SSIM at $1024\times1024$ (Yang et al., 26 Aug 2025).

Taken together, these case studies show that MLP-based visual modeling spans at least three distinct operational modes: discriminative classification over engineered or transformed descriptors, end-to-end visual backbones for recognition and restoration, and coordinate-to-signal regressors for continuous representations. This suggests that the family is best understood as a reusable computational substrate whose role changes with the surrounding representation.

5. Training regimes, efficiency, and scaling behavior

A persistent empirical theme is that MLP-based visual models are unusually sensitive to training protocol. ResMLP explicitly attributes its competitiveness to modern, data-efficient training: LAMB with learning rate $^2$ 0, weight decay 0.2, 400 epochs, and DeiT-style augmentations including RandAugment, Mixup, CutMix, Random Erasing, stochastic depth, and repeated augmentation. Under an “old-fashioned” 90-epoch recipe, ResMLP-S12 drops to 69.2%, whereas the reported modern-training result is 76.6% (Touvron et al., 2021). Vision Permutator uses AdamW with weight decay $^2$ 1, batch size 2048, 300 epochs, and strong augmentation, again with no positional encodings (Hou et al., 2021). DynaMixer likewise relies on AdamW, 300 epochs, strong augmentations, LayerNorm, residual connections, stochastic depth, and mixed precision on $^2$ 2 NVIDIA A100 GPUs (Wang et al., 2022).

The second recurring theme is the cost of naive global token mixing. The survey states that token-mixing MLPs incur parameter complexity $^2$ 3 and FLOPs $^2$ 4, which ties them to fixed spatial resolutions and makes high-resolution scaling difficult (Liu et al., 2021). This is the core reason many later models use axial factorization, shifts, local windows, or hierarchical pyramids. ConvMLP makes the trade-off explicit by replacing fixed-resolution spatial MLPs with resolution-agnostic convolutions and per-location channel MLPs; ConvMLP-S reports 76.8% top-1 on ImageNet-1k with 9.0M parameters and 2.4 GMACs, and its hierarchical feature pyramid transfers directly to RetinaNet, Mask R-CNN, and Semantic FPN (Li et al., 2021).

Efficiency is not only about asymptotic complexity but also about operator format. BiMLP investigates binary MLPs under the observation that fully connected layers can be treated as $^2$ 5 convolutions, and that binarization sharply limits representational capacity. Its unified operation metric is $^2$ 6, yielding 0.156G OPs for BiMLP-S and 0.188G OPs for BiMLP-M, with Top-1 accuracies of 70.0% and 72.7% on ImageNet-1k (Xu et al., 2022). MAXIM addresses a different efficiency constraint: high-resolution restoration. Its multi-axis gMLP block has complexity linear in the number of pixels $^2$ 7, and the architecture remains fully-convolutional and resolution-agnostic, avoiding the fixed-token limitations of patch-based mixers (Tu et al., 2022).

These design choices reflect a common systems-level compromise. Pure global MLP mixing offers conceptual simplicity but poor scaling, whereas efficient MLP-based visual models usually introduce axis factorization, hierarchy, windowing, or hybrid stems. This suggests that the competitive zone of MLP-based vision is defined less by operator purity than by how effectively a model reshapes global linear mixing into tractable structured computation.

6. Limitations, debates, and prospective directions

The central debate surrounding MLP-based visual models is whether they constitute a distinct paradigm or a transient reformulation of mechanisms already better expressed by convolution and attention. The survey is cautious: token-mixing MLPs suffer from overfitting risk, quadratic cost in image resolution, and resolution sensitivity, and MLP backbones generally trail the strongest CNN and Transformer baselines on detection and segmentation (Liu et al., 2021). Even on classification, the best reported no-extra-data ImageNet-1k numbers in the surveyed family remain around the low-to-mid 84% range rather than matching the strongest large-scale CNN or Transformer systems.

A related controversy concerns operator purity. Many successful “MLP-based” models reincorporate locality, hierarchy, or auxiliary convolutional mechanisms. Hire-MLP introduces spatial inductive bias through hierarchical rearrangement (Guo et al., 2021). ConvMLP openly uses convolutional tokenization, depthwise convolutions, and convolutional downsampling (Li et al., 2021). MAXIM is MLP-centric in its core blocks but augments them with RCAB modules (Tu et al., 2022). This suggests that the practical trajectory of the field has been toward hybridization, even when the rhetoric emphasizes all-MLP design.

Application-specific limitations are equally clear. The log-polar face-recognition pipeline is sensitive to center selection and does not directly address occlusion or illumination (Bhowmik et al., 2010). ViP’s axis-specific projections create fixed-shape constraints that complicate variable-resolution dense prediction (Hou et al., 2021). DynaMixer assumes fixed input size in its reported experiments and incurs generator overhead relative to static mixers (Wang et al., 2022). R $^2$ 8-MLP depends on a meaningful cyclic view ordering, so irregular or misordered view sets weaken its inductive bias (Chen et al., 2022). T-MLP requires careful tuning of the $^2$ 9 weights because improving one LoD can degrade another, and its first tail is typically too weak to supervise directly (Yang et al., 26 Aug 2025). BiMLP still leaves first, last, and downsampling layers in full precision, which indicates that end-to-end binarization remains unresolved (Xu et al., 2022).

The future directions in the literature reflect these fault lines. The survey calls for MLP-specific pretraining, optimization, compression, interpretability, and hardware co-design (Liu et al., 2021). DynaMixer points toward content-adaptive mixing without attention (Wang et al., 2022). MAXIM suggests a universal axis-wise perspective in which gMLP, self-attention, and Fourier filters can be inserted into the same multi-axis scaffold (Tu et al., 2022). MC-MLP proposes extending spatial mixing into multiple coordinate frames through fixed orthogonal transforms (Zhu et al., 2023). T-MLP indicates that depth itself can become a semantic or LoD axis rather than merely a feature-extraction hierarchy (Yang et al., 26 Aug 2025). A plausible implication is that the long-term significance of MLP-based visual models may lie less in displacing convolution or attention wholesale than in providing a flexible linear-mixing vocabulary from which new hybrids, structured projections, and signal representations can be built.