Parameter-efficient Quantum Multi-task Learning

Abstract: Multi-task learning (MTL) improves generalization and data efficiency by jointly learning related tasks through shared representations. In the widely used hard-parameter-sharing setting, a shared backbone is combined with task-specific prediction heads. However, task-specific parameters can grow rapidly with the number of tasks. Therefore, designing multi-task heads that preserve task specialization while improving parameter efficiency remains a key challenge. In Quantum Machine Learning (QML), variational quantum circuits (VQCs) provide a compact mechanism for mapping classical data to quantum states residing in high-dimensional Hilbert spaces, enabling expressive representations within constrained parameter budgets. We propose a parameter-efficient quantum multi-task learning (QMTL) framework that replaces conventional task-specific linear heads with a fully quantum prediction head in a hybrid architecture. The model consists of a VQC with a shared, task-independent quantum encoding stage, followed by lightweight task-specific ansatz blocks enabling localized task adaptation while maintaining compact parameterization. Under a controlled and capacity-matched formulation where the shared representation dimension grows with the number of tasks, our parameter-scaling analysis demonstrates that a standard classical head exhibits quadratic growth, whereas the proposed quantum head parameter cost scales linearly. We evaluate QMTL on three multi-task benchmarks spanning natural language processing, medical imaging, and multimodal sarcasm detection, where we achieve performance comparable to, and in some cases exceeding, classical hard-parameter-sharing baselines while consistently outperforming existing hybrid quantum MTL models with substantially fewer head parameters. We further demonstrate QMTL's executability on noisy simulators and real quantum hardware, illustrating its feasibility.

• The paper proposes a quantum multi-task head that scales linearly with the number of tasks, achieving over 12× parameter reduction compared to classical heads.
• It introduces a variational quantum circuit with a shared encoder and task-local subcircuits to efficiently specialize task outputs.
• Empirical evaluations on NLP, medical imaging, and multimodal tasks demonstrate competitive performance and robustness on actual quantum hardware.

Parameter-efficient Quantum Multi-task Learning: Design, Analysis, and Benchmark Evaluation

Introduction and Motivation

The paper "Parameter-efficient Quantum Multi-task Learning" (2604.13560) addresses one of the central limitations in multi-task learning (MTL): the rapid growth of task-specific parameters with the number of tasks in classical architectures, particularly under hard-parameter-sharing regimes. The work systematically explores whether quantum machine learning (QML), leveraging variational quantum circuits (VQCs), can yield parameter-efficient MTL heads that maintain sufficient task-specific specialization.

A key innovation is the introduction of a quantum MTL head that performs all task-specific computations fully within the quantum circuit, as opposed to existing hybrid quantum neural network (HQNN) approaches which retain classical task-specific layers. The proposed architecture demonstrates linear scaling in head parameters with the number of tasks ($\mathcal{O}(T)$) compared to the quadratic growth ($\mathcal{O}(T^2)$) in classical heads, under capacity-matched constraints. The approach is validated on standard benchmarks in natural language processing, medical imaging, and multimodal sentiment analysis.

Quantum Multi-task Learning Architecture

The proposed architecture replaces classical task-specific heads (typically linear or MLP layers atop a shared representation) with a parameter-efficient VQC head. The design is structured as follows:

1. Shared Quantum Encoder: The extracted feature vector $\mathbf{Z}$ is embedded into a $Q$-qubit quantum register through a shared, trainable quantum state preparation circuit. This block acts as a quantum analog of the shared classical feature backbone.
2. Task-local Subcircuits: The quantum register is partitioned into disjoint sub-registers, each corresponding to a specific task. Lightweight, shallow, task-local quantum ansatz blocks further process these sub-registers, enabling task adaptation with compact parameterization.
3. Multi-observable Readout:

Task predictions are produced by measuring a set of carefully chosen (possibly commuting) Pauli observables on the respective task sub-register. Critically, the observable set can be calibrated to decouple the number of outputs from the number of qubits.

Figure 1: Proposed Quantum Circuit architecture for multi-task learning. An extracted feature vector $Z$ is embedded into a $Q$-qubit register via a shared trainable state-preparation stage $f_Q(\mathbf{Z})$.

This architecture allows a single quantum state, encoding the shared features, to be specialized in parallel for each task with minimal parameter cost. Each task's sub-register is processed by only a handful of trainable parameters, in contrast to traditional full-dimension classical mapping.

Analytical Parameter Scaling

The paper presents closed-form parameter scaling for both classical and quantum MTL heads, under controlled and capacity-matched constraints. The analysis assumes the latent dimension $d$ grows linearly with the number of tasks $T$, ensuring scalability comparisons reflect realistic representational needs.

Key results include:

• Classical Head Parameters:

Each task head is a linear mapping of size $r \times d$, making the total parameter count $\mathcal{O}(T^2)$0. For $\mathcal{O}(T^2)$1, $\mathcal{O}(T^2)$2.

• Quantum Head Parameters:

The quantum head parameter count is $\mathcal{O}(T^2)$3 (where $\mathcal{O}(T^2)$4 is the total qubits, $\mathcal{O}(T^2)$5 and $\mathcal{O}(T^2)$6 are encoding and head depths, etc.), which simplifies to $\mathcal{O}(T^2)$7 for fixed circuit hyperparameters.

Figure 2: Total number of trainable parameters in the proposed quantum MTL architecture versus the classical counterpart.

Empirically, QMTL's head exhibits >12× parameter reduction compared to classical heads and 4-5× compared to HQNNs with equivalent qubit budgets.

Figure 3: Total trainable parameters in task-specific heads across datasets. QMTL achieves more than $\mathcal{O}(T^2)$8 reduction compared to classical, and 4-5$\mathcal{O}(T^2)$9 compared to HQNN.

Empirical Evaluation Across Benchmarks

GLUE (NLP Benchmark)

On the multi-task GLUE NLP suite, QMTL matches or exceeds the classical hard-parameter-sharing baseline on several tasks, including MRPC, QQP, and QNLI.

Figure 4: Performance with respect to primary metrics (GLUE); QMTL matches classical baseline on strong tasks.

Notably, both classical and QMTL heads are significantly more parameter-efficient than HQNNs, which consistently underperform due to feature compression and limited quantum expressivity in those architectures.

CheXpert (Medical Imaging)

On five simultaneous chest pathology classification tasks, QMTL matches or slightly trails the classical head in mean accuracy and F1, distinctly outperforming HQNN baselines regardless of qubit count.

Figure 5: Multi-task performance on the CheXpert dataset across MTL head variants.

Performance variability (fold-to-fold) is not inflated relative to classical baselines, supporting the claim that parameter reduction does not induce instability.

Extended MUStARD (Multimodal)

For five multi-label and multi-class multimodal tasks (sarcasm, two sentiment, two emotion), QMTL is consistently competitive, winning on key binary and ternary tasks and matching the classical baseline on the high-cardinality (9-class) emotion tasks.

Figure 6: Multi-task performance on the Extended MUStARD dataset; QMTL head is robust and competitive.

Further, increasing qubit count in HQNN heads does not improve HQNN performance, underscoring that expressivity in QMTL arises from its shared encoding and task-local specialization structure, not merely circuit size.

Ablation Studies and Analysis

The ablation studies provide empirical evidence on several design axes:

• Parameter Budget: Increasing encoding depth ($\mathbf{Z}$0) improves multi-task accuracy until saturation; increasing task-local ansatz depth ($\mathbf{Z}$1) yields diminishing returns, validating the parameter-efficient allocation.

  Figure 7: Effect of increasing parameter cost on model performance: gains from encoding depth, saturation for task-local head depth.

• Shared Encoder Entanglement: Including entanglement (fixed CNOTs in the shared encoder) provides consistent performance improvement without parameter cost, supporting the role of entanglement for enhanced representational capacity in quantum shared encoders.

  Figure 8: Effect of entanglement in the shared encoder on QMTL performance across CheXpert tasks; entanglement boosts multi-task accuracy.

Hardware Execution and Noise Robustness

QMTL is evaluated under realistic noise via Qiskit’s AerSimulator depolarizing noise models and then on IBM Quantum hardware. Performance degrades gracefully with noise but QMTL still retains task-discriminative power on hardware, and device-to-device differences track error rates.

Figure 9: Multi-task performance of QMTL on CheXpert under increasing depolarizing noise.

Figure 10: Multi-task performance of QMTL on IBM Quantum devices compared to noise-free simulation.

Theoretical and Practical Implications

This study provides clear evidence that fully quantum multi-task heads can achieve statistical parity with classical linear baselines while reducing parameter scaling from quadratic to linear in the number of tasks—a regime that may become critical in high-cardinality or resource-constrained applications (e.g., edge quantum-classical hybrid systems). The approach is compatible with standard quantum hardware and robust up to moderate noise levels.

Key theoretical implications:

• Demonstrates that compact quantum encodings and modular quantum ansatz subcircuits can replicate classical multi-head MTL output mappings with dramatically reduced parameter counts.
• Reinforces that the exponential Hilbert space of the shared quantum encoding can serve as a high-capacity, task-agnostic representation suitable for diverse supervised objectives.

Potential practical directions and future developments include:

• Exploring QMTL for continual and dynamic task regimes, and for task-conditional routing and adaptive MTL architectures without incurring classical parameter inflation.
• Extending QMTL to mixture-of-experts frameworks, error mitigation strategies, and integration with quantum pre-trained models.
• Investigating end-to-end quantum multi-task architectures including quantum-native backbones, particularly as qubit counts and gate fidelities advance.

Conclusion

Parameter-efficient Quantum Multi-task Learning (QMTL) sets a new benchmark for MTL head modularity, proving both analytically and empirically that quantum architectures can maintain or exceed classical baseline performance with greatly reduced trainable parameter complexity. Both the theoretical framework and strong empirical results argue for the relevance of quantum approaches as parameter bottlenecks become acute in high-task-count, resource-constrained, and hybrid quantum-classical learning systems. The work provides a principled foundation for future quantum model development as quantum hardware scales.