Partitioned MLP (PMLP) Overview

• PMLP is a paradigm that partitions MLP architectures along feature, channel, spatial, or privacy dimensions to enhance efficiency, interpretability, and secure computation.
• Variants like FF-MLP, SplitMixer, and Propagational MLP demonstrate specialized designs that leverage geometric partitioning, channel slicing, and test-time graph propagation respectively.
• Secure two-party PMLP protocols (e.g., EVA-S2PMLP) utilize cryptographic techniques to enable privacy-preserving collaborative learning with exact nonlinearity handling.

Partitioned MLP (PMLP) encompasses a spectrum of methodologies in which multilayer perceptrons (MLPs) are architecturally or computationally partitioned, either for efficiency, interpretability, privacy, or hybridization with other frameworks. The PMLP paradigm covers several concrete instantiations, including feedforward designs based on geometric partitionings in feature space, split-layer architectures for computational and parameter savings in vision models, two-party secure computation protocols for vertically partitioned data, and the propagational MLP bridge between MLPs and Graph Neural Networks (GNNs).

1. Feedforward PMLP via Geometric Partitioning

Lin et al. proposed an interpretable PMLP architecture—also labeled feedforward MLP (FF-MLP)—that generalizes two-class linear discriminant analysis (LDA) to the multiclass, multimodal setting (Lin et al., 2020). This design constructs the entire network in a closed-form, eliminating iterative backpropagation during training. The architecture has four layers:

• $l_{in}$: input (dimension $N$)
• $l_{1}$: hyperplane layer, $\mathrm{dim}=D_1$
• $l_{2}$: region-isolation layer, $\mathrm{dim}=D_2$
• $l_{out}$: output (classes, $C$)

The construction comprises three stages:

1. Half-space Partitioning ($l_{in} \rightarrow l_1$): For every pair of $G$ Gaussian components ("blobs"), LDA-derived hyperplanes are computed for separation. Weight vectors $N$0 and biases $N$1 encode the separation. Two neurons per hyperplane (one for each half-space) yield $N$2, $N$3.
2. Subspace Isolation ($N$4): Each region in input space (generated by hyperplane intersection) is assigned a unique "signature". Partitioning yields up to $N$5 region neurons. The associated weight matrix $N$6 encodes combinatorial region membership using large negative constants to enforce exclusivity activated by ReLU nonlinearity.
3. Class-wise Subspace Mergence ($N$7): Each identified region neuron maps to exactly one target class via a binary mask $N$8, followed by softmax activation.

When original classes are non-Gaussian, a Gaussian mixture model (GMM) substitutes for each class, and all stages are generalized to treat each component as a separate "blob," allowing the method to approximate arbitrary data distributions.

Empirical evaluation shows FF-MLP matches or exceeds backprop-trained MLP (BP-MLP) test accuracy on synthetic and real datasets, designs in $N$9–$l_{1}$0 sec, and achieves highly interpretable weight matrices, in contrast to the opacity of BP-MLP weights (Lin et al., 2020).

2. Partitioned MLP in Channel and Spatial Domains

In the context of visual recognition, PMLP also refers to channel-wise partitioning to optimize MLP-like models, notably in SplitMixer (Borji et al., 2022). Here, the core technique is to slice the channel dimension into multiple groups and perform channel mixing within each partition independently, reducing parameter count and computational cost.

Partitioning Variants:

• Overlapping Segments (SplitMixer-I): Channels are split into two partially overlapping groups, and in each block only one segment is mixed via a $l_{1}$1 convolution, halving parameter cost for this operation.
• Non-overlapping Segments (SplitMixer-II, III, IV): Channels are divided into $l_{1}$2 disjoint groups. Mixing is done separately on each group, with or without weight sharing.

Spatial mixing is handled through sequential depthwise 1D convolutions (first across width, then height), instead of full $l_{1}$3 depthwise convolutions. This approach further reduces parameter count and FLOPS by a factor of approximately $l_{1}$4 compared to traditional isotropic MLP-mixer and ConvMixer blocks.

Empirical results on CIFAR-10 and CIFAR-100 demonstrate that SplitMixer with PMLP layers delivers nearly state-of-the-art accuracy with significantly fewer parameters and FLOPS; for example, SplitMixer-I achieves 93.9% accuracy on CIFAR-10 with only 0.28M parameters, compared to ConvMixer's 0.59M parameters for a similar accuracy (Borji et al., 2022).

3. Propagational MLP: MLPs with Test-time Graph Propagation

Propagational MLPs (PMLPs) form an intermediate model class between vanilla MLPs and GNNs. A PMLP is trained exclusively as a standard MLP, but at inference (test) time, one or more nonparametric message-passing (MP) layers are inserted to endow the MLP with inductive graph bias (Yang et al., 2022). For node-level prediction, this scheme demonstrates that graph propagation at inference is responsible for most of the generalization benefits of GNNs, rather than joint training of MP and feedforward layers.

PMLP Construction:

• Training: $l_{1}$5, $l_{1}$6 for $l_{1}$7.
• Test: Apply a propagation operator $l_{1}$8 (e.g., normalized adjacency matrix) to final representation: $l_{1}$9.
• Architectural Variants: Insert $\mathrm{dim}=D_1$0 after every layer (to mimic GCN), propagate for $\mathrm{dim}=D_1$1 steps before a small MLP (SGC), or perform personalized propagation (APPNP).

Theoretical analysis in the infinite-width (NTK) regime confirms that PMLP shares the same kernel feature map and minimum-norm solution as an MLP during training; at inference, the output is transformed by the GNN's NTK feature map. Empirical results on 16 standard node-classification benchmarks show PMLP consistently improves over vanilla MLP (by 20–30% accuracy) and generally matches or slightly exceeds the test accuracy of its GNN analog, with training time 5–65× faster (Yang et al., 2022).

4. Secure Two-Party Partitioned MLP for Vertically Partitioned Data

In privacy-preserving collaborative learning, PMLP refers to vertically partitioned MLP computation, where multiple data owners (e.g., Alice and Bob) possess disjoint feature subsets of the same samples (Peng et al., 18 Jun 2025). EVA-S2PMLP is a modular protocol suite for secure two-party MLP training and inference over real-valued data, supporting exact nonlinearities.

The framework introduces a spatial-scale transformation pipeline that maps scalars into vectors/matrices, enabling secure computation via additive secret sharing and two-party inner-product protocols. Every atomic operation (matrix-matrix multiplication, ReLU, softmax, Hadamard product, softmax reciprocal) is implemented as a secure primitive with explicit correctness and security proofs. The composite protocol supports exact end-to-end gradient descent without leaking intermediate activations or features.

Key Protocols:

• S2PM, S2PRIP: Matrix and inner-product core building blocks.
• S2PRL, S2PDRL: Secure ReLU and ReLU-derivative.
• S2PHM, S2PHP, S2PHHP: Matrix, Hadamard, and hybrid pointwise products.
• S2PSM, S2PSCR: Softmax and scale-collapse reciprocal.
• S2PG-MLP: Secure MLP gradient.

Experimental results show up to $\mathrm{dim}=D_1$2 lower communication than prior art (ABY³, CrypTen), negligible accuracy loss, and $\mathrm{dim}=D_1$3-round per-layer computation (Peng et al., 18 Jun 2025). The design is suitable for finance, healthcare, and cross-organization AI, limited to semi-honest threat models and two-party settings.

5. Comparison of Partitioned MLP Approaches

Variant	Domain	Partition Type	Main Advantage
FF-MLP (Lin et al., 2020)	General tabular/data	Feature-space (geometry)	Interpretable, closed-form, no BP
SplitMixer (Borji et al., 2022)	Vision (CIFAR)	Channel/Spatial	Low parameter/FLOPs, efficient mixing
Propagational (Yang et al., 2022)	Graph/node-level	Train/inference (graph)	GNN-level performance, fast training
EVA-S2PMLP (Peng et al., 18 Jun 2025)	Secure, federated	Feature (vertical)	Confidentiality, exact nonlinearity

Each approach targets a distinct use case: FF-MLP for interpretable and analytical MLPs; SplitMixer for efficient vision models; Propagational MLP as a lightweight GNN alternative; EVA-S2PMLP for secure, collaborative machine learning.

6. Applications, Limitations, and Open Directions

PMLPs are applicable across domains: interpretable pattern recognition with explicit partitions (Lin et al., 2020), resource-efficient vision architectures (Borji et al., 2022), scalable and efficient node prediction on large graphs (Yang et al., 2022), and privacy-preserving collaborative modeling with strict confidentiality and verification guarantees (Peng et al., 18 Jun 2025).

Limitations are domain- and variant-specific. FF-MLP may incur overhead from a large number of regions in high-modal, high-dimensional data. SplitMixer's degree of partitioning trades off accuracy for efficiency and may be limited in highly entangled representations. Propagational MLPs may be outperformed by GNNs on tasks requiring jointly learned propagation; EVA-S2PMLP covers only semi-honest adversarial models and is restricted to two parties.

A plausible implication is that PMLP principles—partitioning in parameter, spatial, computational, or privacy space—will continue to inform efficient, interpretable, and secure neural network architecture design, with future work likely to address many-party privacy, broader classes of nonlinearities, and further integration with domain-specific inductive biases.