MJKAN: Hybrid Layer Bridging KAN and MLP

• Modulation Joint KAN (MJKAN) is a neural layer that integrates univariate function approximators and FiLM-style modulation to balance KAN flexibility with MLP efficiency.
• It employs Gaussian RBF expansions for localized basis functions and per-feature FiLM modulation, offering improved regression performance and interpretability.
• Empirical results show that while MJKAN excels in function regression, careful tuning of the basis size is essential to prevent overfitting in classification tasks.

Modulation Joint KAN (MJKAN) is a Kolmogorov–Arnold-inspired neural layer that combines KAN-style learnable univariate function approximation with MLP-style linear weighting via a FiLM-like mechanism and Gaussian radial basis function (RBF) activations. Introduced in "Bridging KAN and MLP: MJKAN, a Hybrid Architecture with Both Efficiency and Expressiveness" (Joo et al., 7 Jul 2025), it is designed to preserve the strong function-approximation behavior associated with KAN while moving closer to the efficiency, hardware-friendliness, and training behavior of a standard multilayer perceptron. Empirically, the model was validated on function regression, image classification, and text classification, where it showed superior approximation capability on several regression tasks and competitive, though not uniformly superior, behavior on general classification benchmarks.

1. Conceptual origin and motivation

MJKAN emerges from a specific tension in the KAN literature: KANs replace scalar weights with learnable univariate functions, but this increased functional flexibility comes with substantial optimization and systems costs (Joo et al., 7 Jul 2025). A typical KAN layer is written as

$$y_j = \sum_{i=1}^{d_{\text{in}}} a_{ij}\,\phi_{ij}(x_i) + b_j,\quad j=1,\dots,d_{\text{out}},$$

where each edge carries its own learnable univariate map $\phi_{ij}(\cdot)$, usually parameterized by B-splines. The motivation for this design is the Kolmogorov–Arnold representation theorem,

$$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$

which frames multivariate functions as compositions of univariate functions and summation.

The paper identifies three practical limitations of standard KANs. First, they have very high parameter counts and computational cost. Second, they often exhibit inferior performance to MLPs on many general tasks such as vision, NLP, audio, and tabular classification when compared fairly in parameters or FLOPs. Third, training is difficult because spline-based functional weights are high-dimensional and expensive to optimize, and hardware implementations are correspondingly costly. The paper further notes that empirical evidence suggests the activation-function choice is more crucial than the full KAN topology, which weakens the case for carrying the full cost of per-edge spline parameterization.

MJKAN is therefore motivated as a hybrid design with four explicit aims: to keep univariate learnable functions, reintroduce efficient linear weighting similar to MLPs, use kernel expansions instead of heavy B-splines, and add a FiLM-like modulation layer to simplify learning. In this sense, MJKAN is positioned not as a rejection of KAN, but as an attempt to isolate the parts of KAN that are most valuable for approximation while discarding the parts that create the largest practical burden.

2. Layer architecture and modulation mechanism

The core computational object is the MJKANLayer, which maps $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$ to $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$ through

$$\mathbf{y}=\sum_{i=1}^{d_{\text{in}}}\text{FiLM}_i(x_i)+\text{Base}(x).$$

Each scalar input dimension $x_i$ has its own univariate branch, and the paper allows an optional residual term $\text{Base}(x)$, described as a simple linear or MLP-like projection and also as “an optional nonlinear linear residual update” (Joo et al., 7 Jul 2025).

The univariate branch begins with a Gaussian RBF expansion. For each input feature, the layer constructs a $K$-dimensional basis

$$\phi_{ij}(x_i)=\exp\left(-\frac{(x_i-c_j)^2}{2\sigma^2}\right),\quad j=1,\dots,K,$$

where $\phi_{ij}(\cdot)$0 are centers and $\phi_{ij}(\cdot)$1 is a width parameter fixed in the paper. The centers may be fixed on a grid or learned. This basis acts as a localized function space for each scalar feature.

FiLM-like modulation is then applied, but in a self-conditioned form rather than through an external conditioning stream. For each feature $\phi_{ij}(\cdot)$2 and basis index $\phi_{ij}(\cdot)$3, the layer learns vectors

$\phi_{ij}(\cdot)$4

which are aggregated into input-dependent modulation parameters

$\phi_{ij}(\cdot)$5

The resulting per-feature output is

$\phi_{ij}(\cdot)$6

Summing these contributions across input dimensions produces

$\phi_{ij}(\cdot)$7

This architecture is structurally distinct from both parent families. Unlike a standard MLP, there is no global matrix multiply in the primary path; instead, each scalar input generates a full output vector through kernelized, input-dependent affine modulation. Unlike a standard KAN, the model does not learn a separate spline function for every edge. The full dataflow is: compute Gaussian bases for each scalar input, use them to synthesize $\phi_{ij}(\cdot)$8 and $\phi_{ij}(\cdot)$9, form $$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$0, sum across input dimensions, and optionally add $$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$1.

3. Relationship to KANs and MLPs

MJKAN is presented as a conceptual bridge between KAN and MLP rather than a minor modification of either (Joo et al., 7 Jul 2025). On the KAN side, each branch $$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$2 is a univariate function from $$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$3 to $$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$4 constructed through kernel expansion. The paper writes this implicitly as

$$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$5

which shows that each output contribution remains a sum of localized basis responses. The aggregation $$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$6 mirrors the inner summation in the Kolmogorov–Arnold decomposition, while the optional base term is framed as playing the role of an outer function.

On the MLP side, the modulation vectors behave like input-dependent weights and biases. The paper is explicit about the limiting cases. If the modulation parameters are trivial and only the RBF univariate functions remain, the behavior is KAN-like. If the RBF activations degenerate to identity, so that $$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$7 and $$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$8 become constants, the layer reduces to a linear combination akin to an MLP layer, modulo minor parameterization differences. This is the formal basis for the claim that MJKAN “bridges” KAN and MLP.

The expressiveness claim is argued empirically rather than theorematically. The paper does not present new approximation theorems, but it argues that larger RBF bases yield a richer univariate function class and that the resulting sums of nonlinear univariate functions inherit universal approximation capacity in principle. In a multi-layer form, the class logit is written as

$$f(x_1,\dots,x_n)=\sum_{q=0}^{2n}\phi_q\Big(\sum_{p=1}^{n}\psi_{q,p}(x_p)\Big),$$9

with

$\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$0

This explicit decomposition is central to the paper’s interpretability claim: each class logit is represented as a sum of univariate RBF-based polynomials in each feature, structurally closer to KAN than to a conventional dense layer.

4. Empirical performance across regression, vision, and NLP

The empirical evaluation covers function regression, image classification, and text classification (Joo et al., 7 Jul 2025). In function regression, five 1D tasks sampled at 500 points on $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$1 were used: Local Bumps, Global Pattern, Step Function, High-Frequency Sine, and Compositional Function. The baseline was a 2-layer MLP with 128 hidden units, while MJKAN used two stacked MJKANLayer modules plus a linear head with basis sizes $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$2.

The reported RMSEs show that larger basis sizes generally improve approximation. On Global Pattern, MLP128 achieved 0.0840, while MJKAN improved from 1.0631 at $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$3 to 0.0734 at $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$4. On Compositional Function, MLP128 achieved 0.4712, while MJKAN reached 0.2628 at $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$5. On Step Function, the MJKAN RMSE decreased to 0.0638 at $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$6, compared with 0.1179 for MLP128. High-Frequency Sine remained difficult for both models, but MJKAN at $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$7 slightly improved over MLP128, with 0.6918 versus 0.7034. These results support the paper’s claim that representation power scales with basis size in regression settings.

In vision, the setup used fully connected networks with two hidden layers, AdamW, and 10 training epochs. On MNIST, MJKAN reached 96.6% accuracy in 124.57 seconds, versus 97.9% and 120.84 seconds for MLP. On CIFAR-10, MJKAN achieved 50.2% in 127.7 seconds, versus 50.3% in 115.2 seconds for MLP. On CIFAR-100, MJKAN reached 19.2% in 125.3 seconds, compared with 22.7% in 115.5 seconds for MLP. The result on CIFAR-10 is effectively parity, while MNIST and especially CIFAR-100 favor the MLP baseline.

The NLP experiments used frozen SimCSE embeddings from princeton-nlp/sup-simcse-bert-base-uncased, followed by either an MJKAN or MLP classifier. On AG News, MJKAN obtained accuracies of 0.9078, 0.9097, 0.9074, and 0.9020 for basis sizes 5, 10, 25, and 50, respectively, while MLP reached 0.9186. On SMS Spam, MJKAN achieved 0.9815, 0.9815, 0.9790, and 0.9777 across the same basis sizes, while MLP achieved 0.9857. The paper therefore characterizes MJKAN as competitive rather than dominant on text classification.

5. Basis size, generalization, and recurring misconceptions

A central empirical finding is that basis size $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$8 is a capacity-control parameter with opposite effects in regression and classification (Joo et al., 7 Jul 2025). In regression, increasing $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$9 usually lowers RMSE by allowing more flexible univariate functions. In image and text classification, however, large $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$0 can induce severe overfitting.

The vision basis-size ablation makes this especially clear. On CIFAR-10, MJKAN accuracy falls from 50.2% at basis size 5 to 45.3% at 10, 42.4% at 25, and 40.0% at 50. On CIFAR-100, the drop is sharper: 19.2% at 5, 14.2% at 10, and 2.4% at both 25 and 50. The authors interpret this through decision-boundary geometry: large $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$1 permits highly curved boundaries that can wrap tightly around training samples, which is particularly problematic when per-class data are limited.

One common misconception is that larger basis size should monotonically improve MJKAN because it increases expressiveness. The reported evidence contradicts this for mainstream classification. Another misconception is that a KAN-derived architecture should automatically outperform MLPs on general tasks. The paper instead shows a more constrained picture: MJKAN is highly effective as a scalable univariate approximator, but on standard image and text classification it is best understood as competitive under small basis sizes and vulnerable to overcapacity under large ones.

The paper uses a Fourier-series analogy to describe this regime dependence. Small $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$2 corresponds to smoother, lower-complexity approximations, while large $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$3 introduces higher-frequency, “wiggly” behavior. The practical guidance follows directly: larger $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$4 is beneficial for regression or symbolic tasks requiring precise function modeling, whereas image and text classification require small $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$5 or additional regularization.

6. Efficiency, limitations, and position within the KAN literature

The efficiency argument for MJKAN is comparative rather than absolute (Joo et al., 7 Jul 2025). Full KAN layers have parameter and FLOP cost proportional to $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$6, far larger than the $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$7 scaling of an MLP. MJKAN reduces this burden by requiring only $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$8 RBF evaluations per input dimension, independent of $\mathbf{y}\in\mathbb{R}^{d_{\text{out}}}$9, while keeping FiLM parameters that scale with output width. The result is a model that remains somewhat heavier than a plain MLP but substantially simpler than a spline-per-edge KAN. Empirically, on the reported vision benchmarks, the training-time overhead relative to MLP is modest: roughly 8–10 seconds over 10 epochs.

Its limitations are equally explicit. MJKAN does not consistently outperform MLPs on general classification tasks; it is sensitive to basis size; and it still has more architectural complexity than a plain MLP. These constraints define its natural application regime. The paper’s practical guidance is to prefer MJKAN when strong 1D function approximation and per-feature interpretability are important but full KAN cost is prohibitive, especially in regression, symbolic, or low-dimensional structured settings. MLPs remain preferable when the primary objective is maximal accuracy and speed on standard high-dimensional image or text classification.

Within the broader KAN literature, MJKAN belongs to a family of attempts to preserve adaptive basis-function behavior while improving practical usability. A related but distinct development is KANMixer, which asks whether KAN can serve as a new modeling core for long-term time series forecasting and reports state-of-the-art performance in 16 out of 28 experiments across seven benchmark datasets (Jiang et al., 3 Aug 2025). This suggests a broader research trajectory in which KAN-derived models are being specialized for different inductive biases and deployment constraints. A plausible implication is that MJKAN represents the branch of this trajectory most directly concerned with reconciling KAN-style univariate decomposition with MLP-like efficiency and training behavior.