KAConvText: Spline-Based Text Convolution

• KAConvText is a 1D text classification architecture that replaces standard linear convolutions with spline-parameterized univariate functions to capture non-linear n-gram interactions.
• It leverages the Kolmogorov–Arnold Representation Theorem to integrate interpretable feature extraction into convolutional kernels and classifier heads.
• Experiments on Burmese hate speech, news, and language identification tasks show high accuracies up to 99.82% and strong weighted F1 scores.

Searching arXiv for the cited papers to ground the article in the current literature. Tool call: arxiv_search({"query":"id:(Thu et al., 9 Jul 2025) OR id:(Drokin, 2024)","max_results":5,"sort_by":"relevance"}) Searching individually to confirm metadata and ensure the discussion is anchored to the relevant arXiv records. Tool call: arxiv_search({"query":"(Thu et al., 9 Jul 2025)","max_results":3,"sort_by":"relevance"}) Tool call: arxiv_search({"query":"(Drokin, 2024)","max_results":3,"sort_by":"relevance"}) KAConvText, short for Kolmogorov–Arnold Convolution for Text, is a 1D text-classification architecture that adapts Kolmogorov–Arnold Networks (KANs) to convolutions over token embeddings by replacing the standard linear dot-product kernel with learnable spline-parameterized univariate functions. It was introduced as the first application of Kolmogorov-Arnold Convolution for Text in sentence classification and evaluated on Burmese hate speech detection, news classification, and ethnic language identification. In the reported experiments, the best configuration—KAConvText–MLP with fine-tuned fastText embeddings—achieved 91.23% accuracy with weighted F1 of 0.9109 for hate speech detection, 92.66% accuracy with weighted F1 of 0.9267 for news classification, and 99.82% accuracy with weighted F1 of 0.9982 for language identification (Thu et al., 9 Jul 2025).

1. Conceptual basis and relation to earlier KAN convolutions

KAConvText is motivated by the Kolmogorov–Arnold Representation Theorem (KART), which is used as the theoretical backbone for capturing high-dimensional non-linear interactions through compositions of univariate functions and sums. In KAConvText, this principle is operationalized by designing the convolutional kernel as a set of learnable univariate spline functions acting on local windows of token embeddings, aggregating the resulting responses through sums and pooling, and then passing pooled features to either a standard MLP head or a KAN head (Thu et al., 9 Jul 2025).

Its distinction from adjacent architectures is structural rather than merely notational. A standard CNN uses linear kernels with fixed nonlinearities such as ReLU, so local pattern extraction is driven by a linear operator followed by a pointwise activation. CNN–KAN hybrids retain standard CNN convolutions for feature extraction and replace only the final linear classifier with a KAN layer, which improves interpretability at classification while leaving feature extraction opaque. KAConvText instead relocates the KAN principle into the convolution itself: the kernel becomes a bank of spline-parameterized univariate functions, and the classification head may then be either an MLP for efficiency or a KAN layer for end-to-end interpretability (Thu et al., 9 Jul 2025).

Within the broader literature, convolutional KANs were studied first in computer vision, where design variants included splines, RBFs, wavelets, and polynomial parameterizations, together with bottleneck and mixture-of-experts formulations. That literature provides the methodological context for treating convolution as additive aggregation over learned univariate functions, although the NLP instantiation in KAConvText is specifically a text adaptation evaluated on Burmese sentence classification rather than a direct reuse of the vision architectures (Drokin, 2024).

2. Mathematical formulation

The paper uses the following canonical form of KART:

$$f(x_1,\dots,x_n) = \sum_{q=1}^{2n+1} \phi_q\!\left( \sum_{p=1}^{n} \psi_{q,p}(x_p) \right).$$

This form motivates replacing fixed-weight linear layers by compositions of learnable univariate functions and sums, and it also motivates rethinking the convolution kernel as a learnable bank of univariate functions (Thu et al., 9 Jul 2025).

For contrast, a baseline 1D CNN over embeddings $X \in \mathbb{R}^{L \times d}$ with kernel width $k$ and output channel $c$ computes

$$H_c[t] = \mathrm{ReLU}\!\left( \sum_{m=1}^{k} \sum_{n=1}^{d} W_{c,m,n} \, X_{t+m-1,n} + b_c \right).$$

KAConvText replaces this linear dot product with position- and dimension-specific univariate functions. For each output channel $c$, kernel offset $m$, and embedding dimension $n$, the kernel element is defined as

$$\phi_{c,m,n}(x) = w^{(c,m,n)}_{\mathrm{s}} \cdot \mathrm{spline}(x) \;+\; w^{(c,m,n)}_{\mathrm{b}} \cdot b(x),$$

where

$$\mathrm{spline}(x) = \sum_{r=1}^{G} \alpha_r B_r^{(k)}(x)$$

is a linear combination of B-spline basis functions of order $X \in \mathbb{R}^{L \times d}$0 over a grid of size $X \in \mathbb{R}^{L \times d}$1, and $X \in \mathbb{R}^{L \times d}$2 is a fixed base function; the paper uses a PReLU-like base, described as a parametric linear piecewise function. The KAConv forward pass is

$X \in \mathbb{R}^{L \times d}$3

Sequence-level aggregation is performed through adaptive average pooling:

$X \in \mathbb{R}^{L \times d}$4

Two classifier heads are then defined. KAConvText–MLP uses

$X \in \mathbb{R}^{L \times d}$5

whereas KAConvText–KAN maps pooled features to logits with spline functions in the classifier head before applying softmax (Thu et al., 9 Jul 2025).

The training objective is standard cross-entropy, and evaluation uses Accuracy and weighted F1. For class-imbalanced settings, weighted F1 is emphasized. The paper does not state the use of focal loss or class weighting in the loss (Thu et al., 9 Jul 2025).

3. Architecture, variants, and implementation details

The shared backbone comprises three convolutional blocks with channel sizes 64, 128, and 256, and kernel sizes 3, 4, and 5, respectively. Adaptive average pooling precedes the classifier, and dropout is fixed at 0.3. In the CNN and CNN–KAN baselines, each convolution is a standard 1D convolution followed by ReLU. In KAConvText, the kernels use cubic B-splines with order $X \in \mathbb{R}^{L \times d}$6, and Instance Normalization plus GELU is applied before the base convolution in KAConv blocks (Thu et al., 9 Jul 2025).

The four evaluated model variants are differentiated by where the Kolmogorov–Arnold parameterization is applied. CNN uses a standard 1D convolutional feature extractor and a linear softmax classifier. CNN–KAN keeps the same feature extractor but replaces the classifier with a KAN layer. KAConvText–MLP replaces the convolutional kernels with spline-parameterized univariate functions and uses an MLP plus softmax classifier. KAConvText–KAN combines a KAConv feature extractor with a KAN classifier head, so interpretability extends to both feature extraction and classification (Thu et al., 9 Jul 2025).

The default KAConv/KAN hyperparameters are grid size $X \in \mathbb{R}^{L \times d}$7, spline order $X \in \mathbb{R}^{L \times d}$8, scale_noise $X \in \mathbb{R}^{L \times d}$9, scale_base $k$0, scale_spline $k$1, grid_eps $k$2, and grid_range $k$3. For KAConv, stride $k$4, padding $k$5, dilation $k$6, and groups $k$7. Training uses Adam with learning rate $k$8 for 10 epochs. No explicit weight decay or early stopping is reported. The implementation is in PyTorch, training is conducted in a Kaggle environment with an NVIDIA P100 GPU with 16 GB memory, and no data augmentation is used in the presented experiments (Thu et al., 9 Jul 2025).

In the wider convolutional KAN literature, alternative bases and bottleneck designs are also studied, including polynomial and wavelet parameterizations and 1×1 squeeze/expand formulations for parameter efficiency. Those design directions provide broader context, but the KAConvText paper evaluates a spline-based text model rather than those vision-oriented variants (Drokin, 2024).

4. Embeddings, corpora, and evaluation tasks

The embedding study compares random initialization against fastText embeddings in both static and fine-tuned settings, with dimensions 100 and 300 and with both CBOW and Skip-gram algorithms. fastText is pre-trained on in-domain corpora and then either frozen or fine-tuned during task training. The paper reports that fine-tuning consistently helps across tasks, that 300-dimensional Skip-gram often yields the best results in KAConvText settings, and that gains from pre-training are substantial even when embeddings remain static (Thu et al., 9 Jul 2025).

Two embedding corpora are used. For hate speech and news classification, a Burmese monolingual corpus contains 341,221 sentences and 7,205,337 tokens; it is cleaned and word-segmented for news and syllable-segmented for hate speech. For language identification, an ethnic multilingual corpus contains 200,781 sentences and 2,755,734 tokens and is syllable-segmented (Thu et al., 9 Jul 2025).

The experimental tasks are all sentence classification problems with dataset splits of 80/20 per class for training and testing, with validation drawn from the 20% according to the paper’s phrasing, and all settings trained for 10 epochs. The hate speech dataset is binary and imbalanced, with 10,140 syllable-segmented sentences from Myanmar social media and public forums; Hate has 8,493 instances (83.76%) and Non-Hate has 1,647 (16.24%), with multiple hate categories collapsed into a single Hate class. The news dataset is balanced and multiclass, with 7,315 word-segmented sentences across six classes: Sports 1,232 (16.84%), Politics 1,228 (16.79%), Technology 1,224 (16.73%), Business 1,221 (16.69%), Entertainment 1,205 (16.47%), and Environment 1,205 (16.47%). The ethnic language identification dataset is imbalanced and multiclass, with 108,016 syllable-segmented sentences across nine languages that mostly share Burmese script: Burmese 19,519 (18.07%), Beik 3,385 (3.13%), Dawei 3,537 (3.27%), Mon 5,854 (5.42%), Pa’o 10,346 (9.58%), Po Kayin 10,031 (9.29%), Rakhine 9,778 (9.05%), S’gaw Kayin 36,300 (33.61%), and Shan 9,266 (8.58%). The paper identifies shared script, syllabic segmentation, and dialectal variety as central challenges in the language identification setting (Thu et al., 9 Jul 2025).

5. Reported performance and computational trade-offs

The best headline results are obtained by KAConvText–MLP with fine-tuned fastText embeddings (Thu et al., 9 Jul 2025).

Task	Best configuration	Reported performance
Hate speech detection	KAConvText–MLP + fine-tuned fastText	91.23% accuracy, F1 = 0.9109
News classification	KAConvText–MLP + fine-tuned fastText	92.66% accuracy, F1 = 0.9267
Language identification	KAConvText–MLP + fine-tuned fastText	99.82% accuracy, F1 = 0.9982

The embedding-wise comparison is more nuanced than the headline figures alone. With random embeddings, CNN is best on hate speech at 88.96% accuracy and 0.8895 F1, KAConvText–MLP is best on news at 89.67% accuracy and 0.8969 F1, and all language-identification models are near ceiling, with CNN and CNN–KAN at F1 0.9973 and KAConvText variants around 0.9971–0.9962. With static fastText, CNN–KAN is best on hate speech at 90.29% accuracy and 0.9017 F1, KAConvText–MLP is best on news at 91.50% accuracy and 0.9147 F1, and KAConvText–KAN is slightly highest on language identification at F1 0.9975, although the differences are described as tiny. With fine-tuned fastText, KAConvText–MLP is best on all three tasks; on hate speech it outperforms CNN at F1 0.9109 versus 0.9023, on news it reaches F1 0.9267 while CNN–KAN and KAConvText–KAN are close behind at approximately 0.918–0.919, and on language identification it reaches F1 0.9982 while CNN and CNN–KAN remain near ceiling at 0.9978–0.9979 (Thu et al., 9 Jul 2025).

The comparison between the two KAConvText heads is consistent across tasks: KAConvText–MLP achieves the highest overall accuracy and F1, whereas KAConvText–KAN is slightly behind but provides interpretability in both the convolutional extractor and the classifier (Thu et al., 9 Jul 2025).

These gains are accompanied by substantial overhead. Under fine-tuned fastText with 300-dimensional embeddings, hate speech models range from CNN at approximately 0.958M parameters, 26.19 s training time, and 0.23 s evaluation time to KAConv–KAN at approximately 2.996M parameters, 207.68 s training time, and 2.69 s evaluation time. For news classification, the range is from CNN at approximately 3.372M parameters, 20.75 s training, and 0.17 s evaluation to KAConv–KAN at approximately 5.419M parameters, 154.74 s training, and 2.00 s evaluation. For language identification, the computational gap widens further: CNN has approximately 4.685M parameters, 663.78 s training time, and 2.51 s evaluation time, whereas KAConv–KAN has approximately 6.739M parameters, 4418.59 s training time, and 30.83 s evaluation time. The reported interpretation is that KAConv variants are more computationally expensive and have larger parameter budgets than CNN and CNN–KAN, trading efficiency for representational power and interpretability (Thu et al., 9 Jul 2025).

6. Interpretability, robustness, limitations, and future directions

A defining property of KAConvText is its claim to transparency in feature extraction. Each convolutional kernel is a univariate function parameterized by B-splines, and the paper visualizes spline surfaces over training epochs. Early training shows sharp spikes associated with exploration; by Epoch 10, the surfaces become smoother with higher overall magnitude, which is interpreted as amplification of useful spline patterns together with flattening of extremes. In this framework, CNN–KAN makes only the classifier interpretable, whereas KAConvText makes feature extraction interpretable as well, especially in the KAConvText–KAN variant (Thu et al., 9 Jul 2025).

The robustness discussion centers on class imbalance. Hate speech and language identification are imbalanced tasks, and weighted F1 is used to emphasize robustness to skew. No explicit focal loss or class-weighted loss is reported, yet KAConvText still attains strong weighted F1. This suggests that the spline kernels help capture minority-class features, although that implication remains interpretive rather than a direct ablation result. The paper also states that KAConvText is designed to model non-linear n-gram interactions and should be applicable to other languages, particularly those with syllabic segmentation or rich morphology (Thu et al., 9 Jul 2025).

Several limitations are stated explicitly. KAConvText incurs higher parameter counts and longer runtimes than CNN and CNN–KAN, particularly on large datasets such as language identification. Performance gains are larger when fastText is fine-tuned at 300 dimensions, indicating sensitivity to embedding quality and domain fit. Hyperparameters such as grid size, spline order, and gating factors affect expressivity and smoothness, so careful tuning may be required. Interpretability also has limits: although kernels and KAN heads are visualizable, attributing decisions to specific tokens still requires careful analysis, and convolutional locality can make attributions position-dependent (Thu et al., 9 Jul 2025).

The broader convolutional KAN literature adds a methodological caution. Convolutional KANs have been explored with multiple basis families and regularization schemes, and KANs have attracted both interest and debate; this underscores that spline-based KAConvText is one specific realization within a wider design space rather than a settled canonical form (Drokin, 2024). The future directions identified for KAConvText are evaluation across more languages and domains, integration with contextual embeddings such as BERT- or LLM-derived features, and extension to pairwise classification tasks including text similarity, paraphrase detection, and natural language inference (Thu et al., 9 Jul 2025).