Towards Fair Benchmarking of Quantum Transfer Learning for Visual Classification

Abstract: Quantum Transfer Learning (QTL) offers a promising approach for visual quantum machine learning under near-term constraints, where limited qubit counts, shallow circuit depths, and costly hybrid optimization restrict end-to-end quantum training. In this setting, pretrained classical backbones can extract high-level visual features, while compact quantum modules operate as trainable classification heads. However, existing QTL results are difficult to compare because they often differ in datasets, preprocessing, backbone settings, qubit budgets, circuit designs, optimization choices, and reporting protocols. This work presents a controlled benchmarking methodology for evaluating representative QTL methods under a unified transfer-learning pipeline. The benchmark compares DQN-QTL, QPIE-QTL, AE-CQTL, PVCQTL, and ED-QTL under shared preprocessing rules, frozen-backbone settings, training conditions, and reporting metrics. The evaluation focuses on Fashion-MNIST and Hymenoptera Ants vs Bees as the two main datasets, while CIFAR-10 is used to provide additional configuration-level evidence on a harder natural-image task. Beyond predictive performance, the benchmark analyzes circuit size, trainable parameters, quantum parameters, training time, and architectural sensitivity to qubit count and circuit depth. The results show that no single QTL family dominates across all settings: performance depends on the dataset, encoding strategy, circuit design, and computational cost. These findings highlight the need for resource-aware QTL evaluation and provide guidance for selecting hybrid quantum-classical transfer models under near-term resource constraints.

• The paper introduces a controlled experimental pipeline standardizing preprocessing, backbone networks, and quantum circuit configurations to ensure fair QTL benchmarking.
• It compares multiple QTL methods across diverse datasets, revealing key trade-offs in predictive accuracy versus computational cost.
• Results show that scaling qubits or circuit depth offers non-universal gains, emphasizing the need for context-sensitive design in hybrid quantum-classical architectures.

Towards Fair Benchmarking of Quantum Transfer Learning for Visual Classification

Introduction

The paper "Towards Fair Benchmarking of Quantum Transfer Learning for Visual Classification" (2605.19417) systematically investigates Quantum Transfer Learning (QTL) paradigms for visual data under realistic near-term hardware constraints. By introducing a controlled experimental pipeline, the authors address chronic issues of benchmarking irreproducibility in QTL: they harmonize dataset preprocessing, backbone networks, input sizes, quantum circuit resource limits, and reporting protocols. This methodological rigor enables a direct and competitive comparison of prominent QTL strategies, emphasizing not only predictive accuracy, but also architectural complexity, qubit and circuit resource scaling, and computational cost. The study's scope centers on two visually and structurally distinct image datasets, Fashion-MNIST and Hymenoptera Ants vs Bees, with supplementary results on CIFAR-10 for configuration-level stress testing.

Controlled Hybrid Quantum-Classical Architecture

QTL models in this benchmark leverage a hybrid architecture, with feature extraction performed by a frozen, pretrained classical backbone (ResNet18). The high-dimensional representation is projected into a quantum-compatible vector, which serves as input to a parameterized quantum circuit (PQC). The circuit output, derived from expectation values of designated observables, is finally mapped to the prediction space via a classical readout layer. This hybrid structure reflects practical constraints in the NISQ setting, offloading complexity to a classical frontend and maintaining a quantum footprint compatible with present quantum hardware limitations.

Figure 1: Schematic of the hybrid quantum-classical learning pipeline, detailing how classical feature extraction is paired with a quantum transfer head and final classical readout.

This architecture is shared among all compared methods, ensuring that differences in downstream performance derive from QTL head configuration—encoding, circuit topology, and readout mechanics—rather than inconsistencies in upstream data representation.

Benchmark Design and Protocol

The benchmark isolates three principal axes of variation: predictive power, computational efficiency, and architectural sensitivity (to qubit count and circuit depth). Protocol standardization is essential: identical pretrained backbones, consistent image preprocessing (size, normalization, augmentation), uniform feature projector depths, and comparable quantum simulation environments are enforced for reproducibility.

The primary datasets reflect diverging visual taxonomies and label granularity: Fashion-MNIST (10-class grayscale apparel) and Ants vs Bees (binary natural images, small N). CIFAR-10 provides an additional multi-class, high-variability, color dataset for assessing configuration robustness. All pipeline stages—backbone output, feature reduction, quantum encoding, variational quantum circuit, and final classifier—are explicitly controlled in experiment.

Figure 2: The controlled protocol pipeline for standardized QTL benchmarking, detailing harmonized data loading, backbone freezing, feature mapping, quantum module variation, and metric reporting.

QTL Methods: Strategies and Variational Ansatz Comparison

The QTL methods compared represent the principal directions in near-term quantum transfer learning design:

• DQN-QTL: Places a basic quantum variational layer with $R_Y$ angle encoding and minimal entanglement atop the backbone; serves as the baseline.
• QPIE-QTL: Expands the encoding expressivity via multi-axis input encoding, using $R_X$, $R_Y$, and $R_Z$ rotations for each qubit—testing richer classical-to-quantum information injection.
• AE-CQTL: Directly amplitude-encodes the full $512$-dimensional feature vector, exploiting $9$-qubit representations to preserve more classical information in quantum state amplitudes.
• PVCQTL: Focuses on post-variational measurement design, comparing modified structured measurement sets and additional variational ansatz layers.
• ED-QTL: Employs a teacher-student distillation paradigm, where quantum students are supervised with softened (ensemble) logit targets from fully classical deep models.

Numerical Results: Dataset-Dependent QTL Performance

The study finds that no QTL method exhibits blanket superiority; empirical performance is acutely sensitive to dataset, quantum encoding, and resource allocation. For Fashion-MNIST, QPIE-QTL and PVCQTL variants reach the top accuracy bracket (F1 scores and accuracy $\approx$88%), with DQN-QTL at $6$ qubits matching their results. On Ants vs Bees, AE-CQTL outperforms variational and post-variational designs, notably achieving $94.99\%$ accuracy and F1, attributed to amplitude encoding's advantage when backbone features are already salient and the label space is low-cardinality.

Figure 3: F1-score comparison for all QTL variants across Fashion-MNIST and Ants vs Bees, illustrating strong dataset dependence and non-monotonic ranking shifts between methods.

Extending to CIFAR-10, where stronger visual variability and class overlap increase task difficulty, DQN-QTL at $6$ qubits leads among the tested configurations, but no QTL architecture in this resource regime matches SOTA classical baselines, consistent with current quantum hardware restrictions.

Performance-Cost Trade-Off and Efficiency Analysis

A crucial contribution is the joint analysis of accuracy and computational expense. The study demonstrates that modest accuracy increments often require a superlinear increase in training time and parameter count, especially when circuit size or depth is enlarged. PVCQTL provides accuracy on par with QPIE-QTL but at an exorbitant simulation cost, whereas AE-CQTL achieves strong performance on Ants vs Bees with relatively lower computation, due in part to efficient amplitude encoding and shallow depth sufficing with discriminative backbone features.

Figure 4: Accuracy versus logarithmic training time for all configurations, visualizing the Pareto frontier and exposing substantial cost for marginal accuracy gains in several QTL lines.

ED-QTL, while computationally inexpensive, only achieves competitive results on Fashion-MNIST, failing to translate teacher knowledge to meaningful improvement on Ants vs Bees.

Qubit and Depth Scaling Behavior

Architectural scaling reveals nontrivial, dataset-specific effects. On Fashion-MNIST, increasing the qubit count from $R_X$0 to $R_X$1 significantly improves DQN-QTL and QPIE-QTL performance, but these improvements vanish or reverse on Ants vs Bees. AE-CQTL exhibits a clear, monotonic gain with circuit depth on Fashion-MNIST and CIFAR-10, but limited additional improvement on the binary Ants vs Bees dataset—implying that when extracted features are already maximally linearly separable, additional nonlinear quantum transformation provides little benefit.

Figure 5: Sensitivity of QTL accuracy to qubit number; gains are observed for Fashion-MNIST with higher qubit allocations, but effect size is substantially dataset-dependent.

Practical Implications and Future Perspectives

These findings have several notable implications for the design and deployment of near-term hybrid quantum-classical visual learners:

• No universal QTL recipe: Method strength is empirically non-transferable between datasets; designers must perform context-sensitive QTL selection, balancing performance against resource cost.
• Scaling resources is not universally effective: Adding qubits or circuit layers can improve performance, but gains are problem-dependent and subject to diminishing or negative returns.
• Amplitude encoding and depth scaling: Amplitude-encoded QTL heads are preferable when hardware supports sufficient qubit count and feature vectors are dense and salient; depth scaling yields improvements only when classical features are not fully separable.
• Distillation and measurement design: Ensemble distillation is not a panacea—teacher-student objectives may underperform unless hyperparameters and student quantum capacity match teacher expressivity. Measurement design can enhance information extraction but incurs simulation overhead.

Theoretical ramifications extend to discussions of quantum data encoding, NISQ expressivity limits, and efficient model selection. Practically, this protocol provides a template for resource-aware benchmarking and exposes critical gaps between quantum simulation studies and scalable hardware realization.

Conclusion

This paper establishes a high-standard controlled benchmarking protocol for QTL in visual classification, furnishing a nuanced and quantitative understanding of trade-offs in hybrid quantum-classical model design (2605.19417). The evidentiary basis provided by harmonic protocol and extensive ablation renders comparative conclusions robust, setting the stage for future advances in QTL architectures, quantum hardware utilization, and the integration of resource-aware design in quantum machine learning pipelines. Future extensions should incorporate hardware execution fidelity, noise robustness, broader task coverage, and more sophisticated teacher-student distillation objectives.