Quantum Orthogonal Neural Networks

• Quantum Orthogonal Neural Networks (QOrthoNNs) are a structured family of hybrid models that implement classical orthogonal neural networks using quantum circuits, ensuring exact orthogonality by design.
• They utilize unary encoding and reconfigurable beam splitter gates to perform quantum circuit-based matrix-vector multiplication, directly emulating classical orthogonal layers.
• Applications span medical imaging, PDE solving via QO-SPINNs, and Bayesian anomaly detection, while challenges include readout overhead and qubit scaling constraints.

Searching arXiv for the core QOrthoNN papers and closely related work. arxiv_search.query({"^^^^^^^^4search_query4 Orthogonal Neural Networks\" OR ^^^^^^^^4all:\4 and Quantum Algorithms for Orthogonal Neural Networks\" OR ^^^^^^^^4all:\4 image classification via quantum neural networks\"","start":^^^^4search_query4^^^^,"max_results":^^^^4all:\4search_query4 Quantum Orthogonal Neural Networks (QOrthoNNs) are a structured family of hybrid quantum-classical neural models in which the linear map of an orthogonal neural-network layer is realized by a parameterized quantum circuit acting on a unary-encoded subspace. In the foundational formulation, a QOrthoNN is not a generic variational quantum classifier over the full Hilbert space, but the quantum counterpart of a classical orthogonal neural network: the trainable parameters are circuit angles, the effective weight matrix is orthogonal by construction, and the circuit’s restriction to the unary basis implements the same matrix-vector multiplication as a classical orthogonal layer (&&&^^^^4search_query4^^^^&&&, &&&^^^^4all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&). Later work specialized this idea to separable physics-informed neural networks for PDEs and to Bayesian anomaly-detection models with orthogonal dense and convolutional blocks (&&&^^^^4 OR all:\4^^^^&&&, Mathur et al., 25 Apr 2025).

^^^^4all:\4^^^^. Conceptual foundation

A classical orthogonal neural network constrains a layer weight matrix to satisfy PRESERVED_PLACEHOLDER_^^^^4search_query4^^^^, or the appropriate rectangular analogue. In this setting, the forward pass retains the usual form PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^, followed by a nonlinearity, but the orthogonality constraint is intended to reduce redundancy, stabilize signal propagation, and mitigate vanishing or exploding gradients (&&&^^^^4search_query4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

QOrthoNNs transfer this idea into a quantum-circuit parameterization. The central observation is that quantum circuits are unitary, and that when one restricts to real amplitudes and to the unary or Hamming-weight-preserving subspace, the effective transformation on encoded classical vectors is orthogonal rather than merely unitary. The resulting model therefore lives intrinsically on the orthogonal manifold: orthogonality is not imposed by projection, singular-value correction, or regularization, but by the circuit construction itself (&&&^^^^4all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

This distinguishes QOrthoNNs from the other quantum method studied alongside them in medical imaging. In that work, one method is a quantum-assisted classical neural network using quantum circuits to estimate inner products inside an otherwise classical dense network; the QOrthoNN is the separate method in which the layer itself is an orthogonal transform implemented by a quantum pyramid circuit (&&&^^^^4all:\4^^^^&&&). It is also distinct from generic hardware-efficient ansätze, because its defining property is an exact correspondence between circuit parameters and orthogonal matrices rather than unrestricted variational expressivity (&&&^^^^4 OR all:\4^^^^&&&).

^^^^4 OR all:\4^^^^. Circuit model, unary encoding, and orthogonal representation

The elementary gate used in the foundational constructions is the two-qubit Reconfigurable Beam Splitter gate,

PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^

Restricted to the one-excitation sector PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^, this is a planar rotation or Givens rotation. Because it preserves Hamming weight, a circuit composed only of $RBS$ gates preserves the unary subspace (&&&^^^^4search_query4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

The input vector is loaded in unary amplitude form. For $x \in \mathbb{R}^d$,

$$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$

where $\ket{e_i}$ is the unary basis state with a single excitation on qubit $i$. The construction uses one qubit per feature. In the medical-image experiments, images of size $28\times 28=784$ are first reduced by PCA to PRESERVED_PLACEHOLDER_^^^^4all:\4search_query4^^^^ or PRESERVED_PLACEHOLDER_^^^^4all:\4all:\4^^^^, because current hardware cannot directly load all pixels (&&&^^^^4all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

The orthogonal layer itself is implemented by a pyramidal arrangement of nearest-neighbor PRESERVED_PLACEHOLDER_^^^^4all:\4 OR all:\4^^^^ gates. For a square PRESERVED_PLACEHOLDER_^^^^4all:\4 OR all:\4^^^^ layer, the circuit depth is PRESERVED_PLACEHOLDER_^^^^4all:\44^^^^, and the number of trainable angles is

PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^5

which matches the number of degrees of freedom of an PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^6 orthogonal matrix. For a rectangular PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^7 transform, the parameter count is

PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^8

again exactly matching the degrees of freedom of an orthogonal PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^9 matrix (&&&^^^^4search_query4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

The representation claims are strong. For square layers, any PRESERVED_PLACEHOLDER_^^^^4 OR all:\4search_query4^^^^ orthogonal matrix corresponds to a unique set of pyramid angles, and conversely the circuit determines such a matrix; the same correspondence extends to rectangular layers. In the more detailed exposition, circuits built from PRESERVED_PLACEHOLDER_^^^^4 OR all:\4all:\4^^^^ gates correspond to PRESERVED_PLACEHOLDER_^^^^4 OR all:\4 OR all:\4^^^^, while determinant-PRESERVED_PLACEHOLDER_^^^^4 OR all:\4 OR all:\4^^^^ orthogonal matrices are obtained by appending a PRESERVED_PLACEHOLDER_^^^^4 OR all:\44^^^^ gate on the last qubit (&&&^^^^4all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

^^^^4 OR all:\4^^^^. Forward propagation, readout, and training algorithm

On the unary basis, the quantum circuit implements the orthogonal matrix multiplication directly. If PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^5 is the full unitary defined by the pyramid, then

PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^6

Equivalently, for basis inputs PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^7,

PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^8

so for a general unary-encoded input the output amplitudes are exactly the entries of PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^9 (&&&^^^^4search_query4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

A practical complication is that direct measurement yields probabilities rather than signed amplitudes. The medical-imaging implementation therefore augments the orthogonal layer with one extra control qubit and a fixed loader for the uniform vector. The sign-sensitive output state yields joint probabilities PRESERVED_PLACEHOLDER_^^^^4 OR all:\4search_query4^^^^ and PRESERVED_PLACEHOLDER_^^^^4 OR all:\4all:\4^^^^ satisfying

PRESERVED_PLACEHOLDER_^^^^4 OR all:\4 OR all:\4^^^^

so each signed coordinate is recovered as

PRESERVED_PLACEHOLDER_^^^^4 OR all:\4 OR all:\4^^^^

This is the key readout rule used for hardware inference in the medical-image QOrthoNN experiments (&&&^^^^4all:\4^^^^&&&). More generally, the tomography discussion gives unary-state amplitude estimation with PRESERVED_PLACEHOLDER_^^^^4 OR all:\44^^^^ measurements for PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^5 tomography, and with sign retrieval the overall cost becomes PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^6 (&&&^^^^4 OR all:\4^^^^&&&).

The multilayer model is hybrid rather than fully coherent end to end. For each layer, one classically preprocesses and normalizes the input, quantumly loads it into the unary state, applies the pyramidal circuit, performs tomography or sign-aware readout to recover PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^7, applies the nonlinearity classically, and then reloads the next layer input. In the notation of ordinary feedforward networks,

PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^8

but the linear part PRESERVED_PLACEHOLDER_^^^^4 OR all:\4^^^^9 is realized by the quantum pyramid circuit (&&&^^^^4 OR all:\4^^^^&&&).

Training is performed in angle space rather than in matrix-entry space. The optimization variables are the $RBS$^^^^4search_query4^^^^ angles $RBS$^^^^4all:\4^^^^ defining the pyramid, so orthogonality remains exact throughout training. The inner-timestep formulation writes

$RBS$^^^^4 OR all:\4^^^^

and for a gate acting on coordinates $RBS$^^^^4 OR all:\4^^^^ and $RBS$4 the exact gradient is

$RBS$5

The update is ordinary gradient descent,

$RBS$6

This yields a classical training algorithm, called QPC for Quantum Pyramidal Circuit, with $RBS$7 complexity for the relevant layer-size scaling, compared with $RBS$8 for previously used SVD-based or Stiefel-manifold orthogonal-network training methods (&&&^^^^4search_query4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

A persistent misconception is that QOrthoNNs in these works are trained by parameter-shift on quantum hardware. The medical-image papers explicitly emphasize the opposite point: training can always be performed classically, and the main novelty lies in the exact orthogonal parameterization and its efficient angle-space optimization rather than in hardware gradient estimation (&&&^^^^4all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

4. Medical-image benchmarks and empirical behavior

The first substantial application domain for QOrthoNNs was medical image classification on MedMNIST. Two datasets were used: PneumoniaMNIST, a pediatric chest X-ray binary task with 47^^^^4search_query4^^^^8 training images and 6^^^^4 OR all:\44^^^^ test images, and RetinaMNIST, originally a 5-level diabetic retinopathy task converted into a binary classification problem with ^^^^4all:\4search_query4relevance4search_query4^^^^ training images and 4^^^^4search_query4search_query4^^^^ test images. All images are $RBS$9, but for QOrthoNN the inputs were reduced by PCA to 4 or 8 dimensions because of hardware limits (&&&^^^^4all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

The reported QOrthoNN architectures were $x \in \mathbb{R}^d$^^^^4search_query4^^^^ and $x \in \mathbb{R}^d$^^^^4all:\4^^^^: a single orthogonal layer mapping directly to two outputs, with no hidden layer and a sigmoid activation for binary classification. The comparison involved three modes: SVB, a classical orthogonal neural network trained with singular value bounding; QPC-SIM, the angle-parameterized orthogonal network evaluated in simulation; and QPC-QHW, the same QPC-trained model executed for inference on IBM hardware. Evaluation used AUC and ACC. For non-hardware settings, experiments were repeated ^^^^4all:\4search_query4^^^^ times, with standard deviation typically $x \in \mathbb{R}^d$^^^^4 OR all:\4^^^^ and sometimes up to $x \in \mathbb{R}^d$^^^^4 OR all:\4^^^^; hardware experiments were run once per configuration and took from about 45 minutes to several hours, while simulator or classical training took seconds (&&&^^^^4all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

On PneumoniaMNIST, the $x \in \mathbb{R}^d$4 configuration produced test AUC and ACC of $x \in \mathbb{R}^d$5 for SVB/SVB, $x \in \mathbb{R}^d$6 for QPC-SIM/QPC-SIM, and $x \in \mathbb{R}^d$7 for QPC-SIM/QPC-QHW. For $x \in \mathbb{R}^d$8, the corresponding test results were $x \in \mathbb{R}^d$9, $$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$^^^^4search_query4^^^^, and $$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$^^^^4all:\4^^^^. On RetinaMNIST, the $$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$^^^^4 OR all:\4^^^^ configuration yielded $$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$^^^^4 OR all:\4^^^^ for SVB/SVB, $$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$4 for QPC-SIM/QPC-SIM, and $$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$5 for QPC-SIM/QPC-QHW; the $$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$6 configuration yielded $$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$7, $$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$8, and $$\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},$$9 respectively (&&&^^^^4all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

These benchmarks support several narrow conclusions. First, simulation performance of the QPC parameterization is comparable to classical orthogonal baselines on the small tasks tested. Second, hardware inference is feasible for very small orthogonal models, and on PneumoniaMNIST the simulator-to-hardware degradation can be modest, particularly for $\ket{e_i}$^^^^4search_query4^^^^. Third, the harder Retina task is more sensitive to hardware noise. Fourth, the paper’s other quantum method, the quantum-assisted dense network, generally attained somewhat higher accuracies than the orthogonal model on these benchmarks, so QOrthoNNs were not presented as the strongest empirical classifier in that study (&&&^^^^4all:\4^^^^&&&).

The most striking training result appears outside the smallest hardware settings. On RetinaMNIST with a larger simulated orthogonal network $\ket{e_i}$^^^^4all:\4^^^^, the paper reports test accuracy of $\ket{e_i}$^^^^4 OR all:\4^^^^ for the SVB-based algorithm and $\ket{e_i}$^^^^4 OR all:\4^^^^ for the simulated quantum pyramid circuit algorithm. The reported interpretation is that optimization in the circuit-angle landscape can produce substantially different models from direct orthogonal-weight optimization (&&&^^^^4all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

5. Extensions to scientific machine learning and Bayesian anomaly detection

A major later specialization is the Quantum Orthogonal Separable Physics-Informed Neural Network, or QO-SPINN, which embeds quantum orthogonal MLPs inside SPINNs for PDE solving. In this architecture, each separable subnet is a quantum orthogonal network whose hidden linear maps are implemented by Hamming-weight-preserving circuits built from $\ket{e_i}$4 gates, so the effective hidden-layer matrices satisfy $\ket{e_i}$5. The paper attributes a forward-layer complexity of $\ket{e_i}$6 to the quantum matrix-vector multiplication primitive, compared with the classical $\ket{e_i}$7, with the caveat that the numerical results are entirely classical simulations of the quantum circuits (&&&^^^^4 OR all:\4^^^^&&&).

The orthogonality in QO-SPINN is used not only for representation but also for stability and uncertainty quantification. The paper develops Lipschitz bounds for SPINN and for the orthogonal specialization,

$\ket{e_i}$8

and exploits the exact spectral norm ^^^^4all:\4^^^^ of orthogonal hidden layers to adapt spectral-normalized Gaussian-process ideas without an explicit spectral-normalization step. Empirically, QO-SPINN outperformed SPINN and PINN on the ^^^^4 OR all:\4^^^^D and ^^^^4 OR all:\4^^^^D advection-diffusion tasks, performed comparably to SPINN but worse than PINN on ^^^^4all:\4^^^^D Burgers, recovered the Sine-Gordon inverse-problem parameter $\ket{e_i}$9 versus $i$^^^^4search_query4^^^^ for SPINN when the target was $i$^^^^4all:\4^^^^, and showed mixed but often favorable uncertainty-calibration behavior relative to Monte Carlo dropout on Burgers (&&&^^^^4 OR all:\4^^^^&&&).

A separate extension is the Bayesian Quantum Orthogonal Neural Network for anomaly detection in ^^^^4 OR all:\4^^^^D additive-manufacturing data. This work combines Bayesian learning with orthogonal quantum-circuit parameterizations, treating the trainable circuit angles as random variables with a mean-field Gaussian posterior and optimizing the ELBO

$i$^^^^4 OR all:\4^^^^

The proposed models include a feedforward autoencoder with orthogonal quantum-inspired bottleneck layers and a ^^^^4 OR all:\4^^^^D orthogonal convolutional architecture, OrthoConv^^^^4 OR all:\4^^^^D, in which flattened $i$^^^^4 OR all:\4^^^^ filters form a matrix $i$4 with mutual orthogonality across filters (Mathur et al., 25 Apr 2025).

In the feedforward family, Bayesian QFNN attained the best calibration, with ECE $i$5, compared with $i$6 for FNN point estimate, $i$7 for FNN Bayesian, and $i$8 for QFNN point estimate. In the ^^^^4 OR all:\4^^^^D family, Bayesian learning again improved calibration, but orthogonality did not improve the main architecture: ^^^^4 OR all:\4^^^^D-CNN Bayesian achieved ECE $i$9, better than ^^^^4 OR all:\4^^^^D-QCNN Bayesian at $28\times 28=784$^^^^4search_query4^^^^. The paper therefore presents a nuanced conclusion: Bayesian learning consistently improves calibration, while orthogonality helps more clearly in the simpler feedforward setting than in the tested ^^^^4 OR all:\4^^^^D convolutional architecture (Mathur et al., 25 Apr 2025).

The same anomaly-detection study also reports hardware experiments on IBM’s ^^^^4all:\4 OR all:\4^^^^7-qubit Brisbane device using 8-qubit orthogonal circuits. Real-hardware fidelity was lower than simulation but remained substantial: for random inputs, the average fidelity was about $28\times 28=784$^^^^4all:\4^^^^ at $28\times 28=784$^^^^4 OR all:\4^^^^ shots and $28\times 28=784$^^^^4 OR all:\4^^^^ at $28\times 28=784$4; for real voxel data, it was about $28\times 28=784$5 at $28\times 28=784$6 and $28\times 28=784$7 at $28\times 28=784$8. Unary post-selection and readout-error mitigation were used, and although direct orthogonal-layer MSE rose with the fraction of circuits run on hardware, the full reconstructed-output deviation in the hybrid autoencoder pipeline remained around $28\times 28=784$9 when all ^^^^4 OR all:\4^^^^56 component circuits were executed on hardware (Mathur et al., 25 Apr 2025).

6. Limitations, misconceptions, and open problems

Several limitations recur across the QOrthoNN literature. The first is tomography and readout overhead: because nonlinearities are classical, a multilayer QOrthoNN must estimate the output vector of each quantum orthogonal layer before proceeding to the next, which yields PRESERVED_PLACEHOLDER_^^^^4all:\4search_query4search_query4^^^^ cost with sign retrieval in the unary constructions. The second is qubit usage: unary encoding requires one qubit per feature, and the sign-recovery construction adds an extra qubit, so the medical-image demonstrations were restricted to 5-qubit and 9-qubit settings for 4- and 8-dimensional PCA inputs (&&&^^^^4all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

A second common misconception is that these models already demonstrate quantum computational advantage. The unary pyramidal circuits can be simulated classically in PRESERVED_PLACEHOLDER_^^^^4all:\4search_query4all:\4^^^^, and the foundational papers are explicit that the principal gain is an exact orthogonal parameterization with PRESERVED_PLACEHOLDER_^^^^4all:\4search_query4 OR all:\4^^^^ training updates, not an exponential speedup. The later PDE paper makes a quantum complexity claim for the layer primitive, but it also states that the numerical results are classical simulations and that practical crossover depends on high-fidelity hardware, state preparation, and tomography cost (&&&^^^^4search_query4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&, &&&^^^^4 OR all:\4^^^^&&&).

Hardware limitations are equally prominent. The medical-imaging work reports instability over time, sensitivity to recalibration, dependence on the choice of qubit subset, and increasing fragility at 9 qubits. The anomaly-detection study shows that small orthogonal blocks can run with reasonable fidelity on current hardware, but only at small scale and within a larger hybrid pipeline. These results therefore support hardware compatibility rather than large-scale end-to-end quantum deployment (&&&^^^^4all:\4^^^^&&&, Mathur et al., 25 Apr 2025).

Orthogonality itself is also not uniformly beneficial. The medical-image results show competitiveness rather than dominance over classical orthogonal baselines, and the anomaly-detection study explicitly concludes that in the tested ^^^^4 OR all:\4^^^^D convolutional case orthogonality does not help. In QO-SPINNs, the paper notes a possible expressivity-versus-regularity tradeoff, including reduced ability to capture high-frequency modes, linking this to spectral bias and constrained Lipschitz behavior. This suggests that exact orthogonality functions as a strong inductive bias whose usefulness is domain- and architecture-dependent (Mathur et al., 25 Apr 2025, &&&^^^^4 OR all:\4^^^^&&&).

Open problems identified across the papers include richer Bayesian posteriors beyond mean-field Gaussian, scaling beyond unary or Hamming-weight-^^^^4all:\4^^^^ encodings, broader use of higher-Hamming-weight compound layers, systematic comparison of circuit layouts such as pyramid and butterfly, and determining when angle-space optimization yields genuinely superior minima relative to other orthogonal parameterizations. A plausible implication is that the lasting importance of QOrthoNNs may lie less in immediate hardware acceleration than in the combination of exact orthogonality, circuit-native parameterization, and hybrid training methods that transfer between quantum-compatible and purely classical implementations (&&&^^^^4search_query4^^^^&&&, Mathur et al., 25 Apr 2025).