Perceval Challenge: Photonic QML Benchmark

• Perceval Challenge is an open benchmark for photonic quantum machine learning that standardizes evaluation protocols using a hardware-feasible MNIST task.
• It features a two-phase evaluation where 64 teams engage in preliminary tests and 11 finalists access large-scale GPU simulation and QPU hardware.
• The challenge establishes empirical baselines by comparing training accuracy, convergence metrics, and generalization gaps across diverse photonic QML architectures.

Searching arXiv for the specified paper and closely related photonic QML/Perceval references. The Perceval Challenge is an open, reproducible benchmark for photonic quantum machine learning (QML) centered on a hardware-feasible MNIST classification task and designed to establish unified baselines for evaluating how photonic quantum circuits learn and generalize from limited data (Notton et al., 29 Oct 2025). It uses the full ten-digit MNIST problem at $28\times 28$ resolution, reduced to $6\,000$ training and $600$ testing samples balanced across classes, and it compares variational, hardware-native, and hybrid photonic approaches under fixed train/test splits and standardized evaluation scripts. Conducted over more than three months, the challenge proceeded through an open phase with $64$ teams and a finalist phase in which $11$ teams received access to large-scale GPU simulation and execution on the Quandela Ascella photonic QPU, thereby producing what the paper describes as the first unified baseline of photonic machine-learning performance (Notton et al., 29 Oct 2025).

1. Benchmark definition and evaluation protocol

The central objective of the Perceval Challenge is to provide the first open, reproducible benchmark for photonic QML on a hardware-feasible MNIST task and to establish unified baselines (Notton et al., 29 Oct 2025). The dataset remains a full $10$-class classification problem rather than a binary subset, and all comparisons use the same train/test splits even when participants apply classical preprocessing such as PCA. This design isolates architectural and algorithmic differences while preserving a common evaluation substrate.

The evaluation protocol is divided into two phases. Phase I (open) allowed $64$ teams to submit designs and preliminary results from local experiments. Phase II (finalists) selected $11$ teams, which were then granted access to large-scale GPU simulations through Scaleway QaaS and to the Quandela Ascella photonic QPU through a cloud service (Notton et al., 29 Oct 2025). The protocol reports training versus test accuracy, the generalization gap $\Delta = \text{train} - \text{test}$, convergence measured as epochs to plateau, the number of trainable parameters, FLOPS, and, when hardware execution is involved, the number of shots or sample complexity.

This structure makes the challenge more than a leaderboard exercise. By fixing data splits, seeds, and evaluation scripts, it treats photonic QML as a benchmarking problem in the same sense that mature classical ML subfields rely on standardized corpora and reporting conventions. A plausible implication is that the challenge is intended to shift photonic QML away from isolated proof-of-principle demonstrations toward directly comparable empirical baselines.

2. Photonic platform, software stack, and circuit formalism

The hardware target is the Ascella linear-optical processor, described as a photonic QPU with up to $12$ modes, fixed $6\,000$0 beam splitters, thermo-optic phase shifters, deterministic single-photon sources, SNSPD detectors, and active stabilization and transpilation (Notton et al., 29 Oct 2025). The associated software stack is Perceval, which exposes the circuit primitives and simulation back-ends used by challenge participants.

The elementary optical operations are specified explicitly. A phase shifter on mode $6\,000$1 is

$6\,000$2

A two-mode beam splitter on modes $6\,000$3 is

$6\,000$4

A universal interferometer $6\,000$5 is built by Reck or Clements decomposition into $6\,000$6 phase shifters and beam splitters (Notton et al., 29 Oct 2025). This provides the architectural substrate for both generic variational ansätze and more structured hardware-native constructions.

Data encoding is formulated as phase embedding of a classical vector $6\,000$7, where each component is mapped to a phase through a parameterized function such as $6\,000$8, $6\,000$9, or $600$0 after PCA. More formally, the amplitude-encoding layer for $600$1 modes and $600$2 photons is

$600$3

where $600$4 is the creation operator on mode $600$5 (Notton et al., 29 Oct 2025).

For an input Fock state $600$6 on $600$7 modes and final unitary $600$8, the output probabilities are given by

$600$9

where $64$0 is the submatrix selected by the Fock occupations (Notton et al., 29 Oct 2025). This permanent-based structure is the defining computational primitive behind boson-sampling kernels, measurement features, and several hybrid models used in the challenge.

3. Methodological landscape

The challenge encompasses variational, hardware-native, and hybrid approaches, spanning end-to-end trainable photonic neural networks, boson-sampling kernels, photonic convolution layers, surrogate-assisted models, transfer-learning pipelines, and self-supervised variants (Notton et al., 29 Oct 2025). The paper emphasizes that these methodologies exhibit complementary strengths rather than converging on a single dominant design.

Among the variational and end-to-end approaches, Lancelot (UDENN) uses a unitary-dilation encoder with a hybrid $64$1 and CNN head in a $64$2, $64$3 setting, combining $64$4 $64$5 blocks with brickwork convolution circuits and $64$6 trainable parameters, of which $64$7 are optical phases and $64$8 classical. CodeQalibur (Photonic QNN) implements an end-to-end photonic neural network with $64$9, $11$0, using two generic interferometers in Clements form, adaptive scaling $11$1, and a classical MLP, for a total of $11$2 trainable parameters. QuantumTree (GLASE) couples a boson-sampling mesh to a trainable surrogate $11$3 and a CNN encoder, with $11$4, $11$5 and either $11$6 or $11$7 parameters depending on latent size (Notton et al., 29 Oct 2025).

The hardware-native and kernel-oriented methods place more weight on photonic feature extraction with classical back ends. Qool uses a boson-sampling kernel plus SVM with $11$8, $11$9, a fixed beam-splitter/phase-shifter mesh, and no variational photonic parameters. Qubiteers constructs a trainable photonic quantum kernel as a convolutional kernel, using $10$0 modes per $10$1 patch, $10$2 photons, and either Type 1 or Type 2 encoding with a depth-$10$3 BS/PS ansatz. QARADOV proposes a quantum convolution layer, $10$4, over $10$5 patches with $10$6, $10$7, a fixed feature map, and a trainable MZI mesh of depth $10$8. Quantum Naples uses a boson-sampler embedding followed by an MLP, with $10$9, $64$0, two rectangular meshes, measurement to $64$1 features, and an MLP $64$2 (Notton et al., 29 Oct 2025).

Several entries pursue structured hybridization. QLOQroaches (Photonic QCNN) uses a photon-native QCNN with local universal interferometers for convolution, adaptive photon injection for pooling, and a global interferometer for the dense stage. QTX (Quantum-Train) maps a photonic quantum circuit to an MPS and then to CNN weight generation, using two rectangular interferometers and a tensor-network bond dimension $64$3. Solal (Hybrid FE) interleaves phase-shifter encoding and beam-splitter processing with a classical fully connected encoder. QuantumNomad uses a pretrained CNN with a stacked MZI mesh and photon-count head. CodeQalibur (qSSL) introduces a self-supervised backbone with a photonic projector using an encoder-to-representation-to-phase-encoding-to-interferometer pipeline and a symmetry loss $64$4 (Notton et al., 29 Oct 2025).

Taken together, these methods define the challenge’s comparative landscape: fully trainable photonic models seek end-to-end expressivity; hardware-native kernels exploit permanent sampling and optical structure; hybrid models trade direct quantum capacity for parameter efficiency, classical feature extraction, or surrogate modeling.

4. Optimization objectives and training procedures

The standard supervised objective in the challenge is the cross-entropy classification loss,

$64$5

This is the common baseline for architectures that map photonic outputs, hybrid features, or photon-count statistics into $64$6-class predictions (Notton et al., 29 Oct 2025).

The GLASE surrogate-assisted method adds an auxiliary regression term for the surrogate ensemble:

$64$7

with total loss

$64$8

This explicitly couples supervised classification to agreement between the surrogate and photonic observables, and it helps explain why GLASE is reported as highly competitive in simulation (Notton et al., 29 Oct 2025).

The UDENN training procedure alternates updates: classical parameters are optimized by Adam, while optical phases are optimized by SPSA. Transfer-learning pipelines freeze the classical backbone and train only the bosonic layer by Adam with cross-entropy on photon counts. The qSSL variant adopts an InfoNCE objective for positive and negative pairs,

$64$9

These optimization choices reflect the heterogeneous differentiability and simulation costs of photonic models. A plausible implication is that the challenge doubles as a comparison of training regimes under realistic hardware and simulator constraints, not merely of ansatz families.

5. Empirical baselines and comparative findings

The reported baselines show substantial variation across design families, parameter counts, convergence rates, and hardware realizability (Notton et al., 29 Oct 2025). Several methods exceed $11$0 test accuracy, but the benchmark’s broader conclusion states that there is no demonstrable heuristic quantum advantage on the reduced MNIST task.

Among the finalist results, Qool attains $11$1 training accuracy through the SVM and $11$2 test accuracy with the sigmoid kernel. Lancelot (UDENN) reports $11$3 training accuracy for the CNN baseline and $11$4 test accuracy, with approximately $11$5 s per epoch classically versus $11$6 h per epoch in the hybrid setting, converging in $11$7 epochs. CodeQalibur (Photonic QNN) reaches $11$8 training accuracy for the MLP configuration and $11$9 test accuracy in about $\Delta = \text{train} - \text{test}$0 epochs. QuantumTree (GLASE) reaches $\Delta = \text{train} - \text{test}$1 training and $\Delta = \text{train} - \text{test}$2 test accuracy in simulation with $\Delta = \text{train} - \text{test}$3 parameters and $\Delta = \text{train} - \text{test}$4 epochs, but drops to $\Delta = \text{train} - \text{test}$5 test accuracy on the Ascella QPU with $\Delta = \text{train} - \text{test}$6 shots per image (Notton et al., 29 Oct 2025).

The structured and kernel-based approaches are likewise heterogeneous. QLOQroaches (QCNN) reports $\Delta = \text{train} - \text{test}$7 test accuracy for the small $\Delta = \text{train} - \text{test}$8 case and $\Delta = \text{train} - \text{test}$9 on $12$0, both with $12$1 epochs. Qubiteers reports $12$2 training accuracy for the classical CNN and $12$3 test accuracy for the trained $12$4-layer hybrid PK configuration in $12$5 epochs. QARADOV (qconv2d) reports $12$6 training accuracy for conv2d$12$7MLP and $12$8 best quantum test accuracy, with $12$9 m per epoch and $6\,000$00 epochs. QTX (Quantum-Train) reaches $6\,000$01 training accuracy at $6\,000$02 and $6\,000$03 test accuracy with $6\,000$04 parameters over $6\,000$05 epochs. Quantum Naples reports $6\,000$06 training accuracy for PCA MLP and $6\,000$07 test accuracy with $6\,000$08 quantum and $6\,000$09 classical parameters after $6\,000$10 epochs plus hyperparameter optimization. Solal (Hybrid FE) reports approximately $6\,000$11 test accuracy for the classical-only case and approximately $6\,000$12 for the hybrid case, with $6\,000$13 parameters, approximately $6\,000$14 MFLOPS, and $6\,000$15 epochs. QuantumNomad spans $6\,000$16 to $6\,000$17 test accuracy in quantum transfer learning with approximately $6\,000$18–$6\,000$19 optical parameters over $6\,000$20 epochs. qSSL reaches a maximum $6\,000$21 test score for MLP $6\,000$22, using approximately $6\,000$23–$6\,000$24 projector parameters and $6\,000$25 SSL epochs plus $6\,000$26 linear-evaluation epochs (Notton et al., 29 Oct 2025).

The paper’s comparative analysis groups these outcomes into several patterns. Variational QNNs such as CodeQalibur and Lancelot are fully trainable end-to-end but require many shots or substantial simulation time and do not yet match classical performance on MNIST. Hardware-native kernels such as Qool, Qubiteers, QARADOV, and Quantum Naples often exceed $6\,000$27 test accuracy with few quantum parameters but depend on a classical MLP or SVM back end. GLASE is the most competitive method in simulation, but QPU noise reduces its performance from roughly $6\,000$28 to roughly $6\,000$29. QLOQroaches indicates promise for translation invariance but is constrained by small mode counts. Quantum-Train is highlighted as parameter-efficient, with $6\,000$30 parameters for $6\,000$31 accuracy. Transfer learning and self-supervised learning do not outperform classical baselines and illustrate the difficulty of interfacing static bosonic transforms with pretrained feature spaces (Notton et al., 29 Oct 2025).

6. Reproducibility, infrastructure, and significance

A defining characteristic of the Perceval Challenge is that all implementations, notebooks, and simulation and hardware scripts are open-sourced in a single shared repository at https://github.com/Quandela/HybridAIQuantum-Challenge (Notton et al., 29 Oct 2025). The simulation stack includes the Perceval back-ends CliffordClifford2017 for fast sampling, Naive (Ryser), SLOS, and Stepper for gradient back-propagation. Hardware access is provided through the Quandela Ascella QPU via a cloud portal, with $6\,000$32 shots per inference, while Scaleway Quantum-as-a-Service supports accelerated boson sampling and PyTorch training.

The benchmark’s reproducibility mechanisms are concrete: fixed train/test splits, random seeds, and standardized evaluation scripts. These provisions matter because photonic QML results can depend strongly on simulator choice, photon-number truncation, sampling noise, hardware calibration, and hybrid training loops. The challenge therefore treats implementation transparency as part of the scientific contribution rather than as an auxiliary concern.

The broader conclusions drawn in the paper are correspondingly methodological. It states that the challenge establishes the first unified set of performance baselines for photonic QML across thirteen distinct methods, that open collaborative benchmarks encourage methodological convergence and reproducibility, and that the shared codebase and infrastructure provide a foundation for future challenges involving larger datasets, deeper circuits, and integrated hardware-in-the-loop studies (Notton et al., 29 Oct 2025). This suggests that the Perceval Challenge is significant not because it demonstrates a quantum advantage on MNIST, but because it organizes the field around common tasks, common metrics, and directly comparable photonic workflows.