Tensor-Augmented Convolutional Neural Networks: Enhancing Expressivity with Generic Tensor Kernels

Abstract: Convolutional Neural Networks (CNNs) excel at extracting local features hierarchically, but their performance in capturing complex correlations hinges heavily on deep architectures, which are usually computationally demanding and difficult to interpret. To address these issues, we propose a physically-guided shallow model: tensor-augmented CNN (TACNN), which replaces conventional convolution kernels with generic tensors to enhance representational capacity. This choice is motivated by the fact that an order-$N$ tensor naturally encodes an arbitrary quantum superposition state in the Hilbert space of dimension $d<sup>N$, where $d$ is the local physical dimension, thus offering substantially richer expressivity. Furthermore, in our design the convolution output of each layer becomes a multilinear form capable of capturing high-order feature correlations, thereby equipping a shallow multilayer architecture with an expressive power competitive to that of deep CNNs. On the Fashion-MNIST benchmark, TACNN demonstrates clear advantages over conventional CNNs, achieving remarkable accuracies with only a few layers. In particular, a TACNN with only two convolution layers attains a test accuracy of 93.7$\%$, surpassing or matching considerably deeper models such as VGG-16 (93.5$\%$) and GoogLeNet (93.7$\%$). These findings highlight TACNN as a promising framework that strengthens model expressivity while preserving architectural simplicity, paving the way towards more interpretable and efficient deep learning models.

• The paper introduces TACNN, which replaces standard convolution kernels with generic order-9 tensors that model high-order local pixel correlations through multilinear forms.
• On Fashion-MNIST, a single-layer TACNN reaches 93.1%, while a two-layer 64×64 model achieves 93.7%, matching GoogLeNet and surpassing VGG-16 with fewer parameters.
• The results suggest that tensor-kernel depth can improve parameter efficiency, but broader datasets, rigorous expressivity guarantees, interpretability studies, and hardware tests remain necessary.

Motivation and positioning

The paper introduces the tensor-augmented convolutional neural network (TACNN), a shallow architecture that replaces each conventional $L\times L$ convolution kernel with a fully generic order-$N$ tensor. The design is motivated by two observations. First, tensor-network (TN) classifiers, despite their theoretical richness, consistently underperform deep CNNs on classical benchmarks such as Fashion-MNIST; the authors attribute this to a mismatch between the long-range-correlation bias of TNs and the locally dominated structure of classical image data. Second, conventional CNN kernels encode only a single linear pattern per kernel, while correlations among multiple kernels remain implicit during training. A generic tensor kernel, interpretable as an arbitrary quantum state in a Hilbert space of dimension $2^N$, embeds these correlations explicitly into each kernel.

Architecture and expressivity arguments

Each pixel value $x\in[0,1]$ is feature-mapped to $\ket{x} = x\ket{0} + (1-x)\ket{1}$, so a patch of $N=9$ pixels becomes a product state in $(\mathbb{R}^2)^{\otimes 9}$, a 512-dimensional space. Each tensor kernel is a general superposition $$\ket{\psi_j} = \sum_{\mathbf{s}} c_j(\mathbf{s})\ket{\mathbf{s}}$$, and the convolution output is the inner product

$$y_j(\mathbf{r}) = \sum_{\mathbf{s}} c_j(\mathbf{s}) \prod_{k=1}^{N} x_k^{1-s_k}(1-x_k)^{s_k},$$

a multilinear form over all pixels in the patch. Although parameters enter linearly through the amplitudes $c_j(\mathbf{s})$, the multiplicative basis functions make the input–output mapping nonlinear, giving per-layer expressivity beyond a standard linear convolution — the first two of three "theoretical foundations" claimed by the authors.

For multilayer extension, layer outputs are normalized per channel and passed through a sigmoid to restore values in $N$0, then re-encoded as product states for the next layer's inner products. The second-layer output becomes a highly nonlinear function of the original input spanning the full receptive field, constituting the third foundation: expressivity grows substantially with depth.

A caveat worth noting: the claim that generic tensors "encode fully correlated structures beyond fixed-bond-dimension TNs" is correct in principle, but the parameter count per kernel grows exponentially ($N$1 vs. $N$2), and the paper offers no formal approximation-theoretic guarantee that SGD finds states exploiting this capacity — the argument remains empirical.

Numerical results on Fashion-MNIST

Experiments use $N$3 images, $N$4 kernels ($N$5), Adam at learning rate $N$6, batch size 100, cross-entropy loss, no data augmentation or preprocessing beyond normalization, averaged over 5 seeds.

Single-layer results. TACNN outperforms matched-kernel-count CNNs across all counts $N$7, $N$8. The advantage is largest in the few-kernel regime: with one kernel TACNN reaches 89.7% while CNN shows instability with standard deviation up to 0.6%. To match TACNN accuracy, CNNs need roughly $N$9–$2^N$0 more kernels, and above $2^N$1 CNNs never catch up: CNN saturates at 92.5% ($2^N$2) while TACNN saturates at 93.1% ($2^N$3). Notably, CNNs overfit beyond 512 kernels despite having far fewer parameters per kernel, whereas TACNN does not — the authors conjecture that optimized kernel states do not yet span the full 512-dimensional space, leaving redundancy unused. This explanation is plausible but unverified.

Comparison against baselines. A single-layer TACNN with 32–64 kernels already exceeds all published TN-based classifiers on Fashion-MNIST, including Deep TTN (92.4%) and Residual MPS (91.5%), as well as AlexNet (89.9%), K-SVM, and XGBoost. The authors read this as evidence that local correlation capture dominates global entanglement-style modeling on this benchmark.

Two-layer results. Adding a second tensor-convolution layer yields:

Model	Test accuracy
1-layer TACNN (best)	93.1%
2-layer TACNN, 16×16	93.2%
2-layer TACNN, 32×32	93.6%
2-layer TACNN, 64×64	93.7% (mean 93.65% ± 0.09%; best seed 93.79%)
VGG-16 (vanilla)	93.5%
GoogLeNet	93.7%

The headline result is that a two-convolution-layer TACNN matches GoogLeNet and surpasses VGG-16 while using roughly 33.6% fewer parameters than GoogLeNet and about $2^N$4 fewer than VGG-16. The 32×32 model already reaches 93.6% with over an order of magnitude fewer parameters than the best one-layer model, supporting the conclusion that depth is a more efficient scaling axis than width in this architecture. One methodological caution: accuracies are reported as best-over-epochs rather than final-epoch values, which slightly favors both models but especially inflates comparisons against literature numbers obtained under different protocols.

Relation to tensor-network classifiers

The discussion section argues that TACNN occupies a favorable middle ground between TN classifiers and QCNNs. Fully 2-d ansätze such as PEPS can capture rich spatial correlations but suffer from prohibitive cost and inadequate optimization without a guiding Hamiltonian; quasi-1-d structures are cheaper but poorly suited to intrinsically 2-d data. Because TACNN uses only small localized tensors within a CNN scaffold, it avoids both bond-dimension constraints and the optimization pathologies of large TNs. The authors also note that since its kernels correspond to few-qubit quantum states preparable by shallow circuits, the framework could in principle be realized on NISQ hardware as a hybrid quantum-classical scheme, avoiding the deep-circuit requirements of QCNNs — though this remains a proposal, not a demonstrated implementation.

Limitations and open questions

The paper concedes several points implicitly rather than formally. Evaluation is confined to Fashion-MNIST; no CIFAR, ImageNet-scale, or non-vision tasks are tested, so generality claims rest on extrapolation. The comparison to VGG-16 and GoogLeNet inherits numbers from the original Fashion-MNIST paper under potentially different training budgets, and TACNN itself benefits from extended training (800 epochs for two layers) and best-epoch reporting. The absence of overfitting in TACNN at high kernel counts is explained only conjecturally. The interpretability advantage asserted in the abstract — that tensor kernels are more explainable than conventional ones — is argued conceptually but not demonstrated with any analysis of learned kernel structure. Finally, improved optimization schemes for machine-learning-oriented tensor networks without a target Hamiltonian are explicitly left open, as is any empirical validation of the NISQ-oriented hybrid proposal.

Conclusion

TACNN replaces CNN kernels with generic higher-order tensors acting on Hilbert-space-embedded patches, converting each convolution into a multilinear form with exponentially enlarged per-kernel capacity. On Fashion-MNIST, a two-layer instance achieves 93.7%, matching GoogLeNet and exceeding VGG-16 with substantially fewer parameters, and a single layer already surpasses all reported TN-based classifiers. The results substantiate the central thesis that local, high-order correlation capture — rather than global entanglement structure — is the productive direction for physics-inspired architectures on classical data, while leaving broader-domain validation, rigorous expressivity analysis, and hardware realization as open problems.