---
title: Quantum Orthogonal Neural Networks
url: https://www.emergentmind.com/topics/quantum-orthogonal-neural-networks-qorthonns
type: topic
---

# Quantum Orthogonal Neural Networks

Searching arXiv for the core QOrthoNN papers and closely related work.
arxiv_search.query({"search_query":"all:\"Quantum Orthogonal Neural Networks\" OR all:\"Classical and Quantum Algorithms for Orthogonal Neural Networks\" OR all:\"Medical image classification via quantum neural networks\"","start":0,"max_results":10,"sort_by":"relevance","sort_order":"descending"})
Quantum Orthogonal Neural Networks (QOrthoNNs) are a structured family of hybrid quantum-classical neural models in which the linear map of an orthogonal neural-network layer is realized by a parameterized quantum circuit acting on a unary-encoded subspace. In the foundational formulation, a QOrthoNN is not a generic variational quantum classifier over the full Hilbert space, but the quantum counterpart of a classical orthogonal neural network: the trainable parameters are circuit angles, the effective weight matrix is orthogonal by construction, and the circuit’s restriction to the unary basis implements the same matrix-vector multiplication as a classical orthogonal layer [2106.07198][2109.01831][2212.07389]. Later work specialized this idea to separable physics-informed neural networks for PDEs and to Bayesian anomaly-detection models with orthogonal dense and convolutional blocks [2511.12613][2504.18103].

## 1. Conceptual foundation

A classical orthogonal neural network constrains a layer weight matrix to satisfy \(W^\top W = I\), or the appropriate rectangular analogue. In this setting, the forward pass retains the usual form \(y = Wx\), followed by a nonlinearity, but the orthogonality constraint is intended to reduce redundancy, stabilize signal propagation, and mitigate vanishing or exploding gradients [2106.07198][2212.07389].

QOrthoNNs transfer this idea into a quantum-circuit parameterization. The central observation is that quantum circuits are unitary, and that when one restricts to real amplitudes and to the unary or Hamming-weight-preserving subspace, the effective transformation on encoded classical vectors is orthogonal rather than merely unitary. The resulting model therefore lives intrinsically on the orthogonal manifold: orthogonality is not imposed by projection, singular-value correction, or regularization, but by the circuit construction itself [2109.01831][2212.07389].

This distinguishes QOrthoNNs from the other quantum method studied alongside them in medical imaging. In that work, one method is a quantum-assisted classical neural network using quantum circuits to estimate inner products inside an otherwise classical dense network; the QOrthoNN is the separate method in which the layer itself is an orthogonal transform implemented by a quantum pyramid circuit [2109.01831]. It is also distinct from generic hardware-efficient ansätze, because its defining property is an exact correspondence between circuit parameters and orthogonal matrices rather than unrestricted variational expressivity [2212.07389].

## 2. Circuit model, unary encoding, and orthogonal representation

The elementary gate used in the foundational constructions is the two-qubit Reconfigurable Beam Splitter gate,
\[
RBS(\theta)=
\begin{pmatrix}
1 & 0 & 0 & 0\\
0 & \cos\theta & \sin\theta & 0\\
0 & -\sin\theta & \cos\theta & 0\\
0 & 0 & 0 & 1
\end{pmatrix}.
\]
Restricted to the one-excitation sector \(\mathrm{span}\{\lvert 01\rangle,\lvert 10\rangle\}\), this is a planar rotation or Givens rotation. Because it preserves Hamming weight, a circuit composed only of \(RBS\) gates preserves the unary subspace [2106.07198][2212.07389].

The input vector is loaded in unary amplitude form. For \(x \in \mathbb{R}^d\),
\[
\ket{x}=\frac{1}{\|x\|}\sum_{i=1}^{d} x_i \ket{e_i},
\]
where \(\ket{e_i}\) is the unary basis state with a single excitation on qubit \(i\). The construction uses one qubit per feature. In the medical-image experiments, images of size \(28\times 28=784\) are first reduced by PCA to \(d=4\) or \(d=8\), because current hardware cannot directly load all pixels [2109.01831][2212.07389].

The orthogonal layer itself is implemented by a pyramidal arrangement of nearest-neighbor \(RBS\) gates. For a square \(n\times n\) layer, the circuit depth is \(2n-3\), and the number of trainable angles is
\[
N=\frac{n(n-1)}{2},
\]
which matches the number of degrees of freedom of an \(n\times n\) orthogonal matrix. For a rectangular \(n\times d\) transform, the parameter count is
\[
\frac{(2n-1-d)d}{2},
\]
again exactly matching the degrees of freedom of an orthogonal \(n\times d\) matrix [2106.07198][2212.07389].

The representation claims are strong. For square layers, any \(n\times n\) orthogonal matrix corresponds to a unique set of pyramid angles, and conversely the circuit determines such a matrix; the same correspondence extends to rectangular layers. In the more detailed exposition, circuits built from \(RBS\) gates correspond to \(SO(n)\), while determinant-\(-1\) orthogonal matrices are obtained by appending a \(Z\) gate on the last qubit [2109.01831][2212.07389].

## 3. Forward propagation, readout, and training algorithm

On the unary basis, the quantum circuit implements the orthogonal matrix multiplication directly. If \(U_W\) is the full unitary defined by the pyramid, then
\[
U_W\ket{x}=\ket{Wx}.
\]
Equivalently, for basis inputs \(\ket{e_j}\),
\[
\ket{e_j}\mapsto \sum_i W_{ij}\ket{e_i},
\]
so for a general unary-encoded input the output amplitudes are exactly the entries of \(Wx\) [2106.07198][2212.07389].

A practical complication is that direct measurement yields probabilities rather than signed amplitudes. The medical-imaging implementation therefore augments the orthogonal layer with one extra control qubit and a fixed loader for the uniform vector. The sign-sensitive output state yields joint probabilities \(\Pr[0,e_j]\) and \(\Pr[1,e_j]\) satisfying
\[
\Pr[0,e_j]-\Pr[1,e_j]=\frac{W_jx}{\sqrt n},
\]
so each signed coordinate is recovered as
\[
W_jx=\sqrt n\,\big(\Pr[0,e_j]-\Pr[1,e_j]\big).
\]
This is the key readout rule used for hardware inference in the medical-image QOrthoNN experiments [2109.01831]. More generally, the tomography discussion gives unary-state amplitude estimation with \(O(\log(n)/\delta^2)\) measurements for \(\ell_\infty\) tomography, and with sign retrieval the overall cost becomes \(\widetilde{O}(n/\delta^2)\) [2212.07389].

The multilayer model is hybrid rather than fully coherent end to end. For each layer, one classically preprocesses and normalizes the input, quantumly loads it into the unary state, applies the pyramidal circuit, performs tomography or sign-aware readout to recover \(z^\ell\), applies the nonlinearity classically, and then reloads the next layer input. In the notation of ordinary feedforward networks,
\[
a^\ell=\sigma(W^\ell a^{\ell-1}+b^\ell)=\sigma(z^\ell),
\]
but the linear part \(W^\ell a^{\ell-1}\) is realized by the quantum pyramid circuit [2212.07389].

Training is performed in angle space rather than in matrix-entry space. The optimization variables are the \(RBS\) angles \(\theta\) defining the pyramid, so orthogonality remains exact throughout training. The inner-timestep formulation writes
\[
\zeta^{\lambda+1}=w^\lambda \zeta^\lambda, \qquad \delta^\lambda=(w^\lambda)^\top \delta^{\lambda+1},
\]
and for a gate acting on coordinates \(i\) and \(i+1\) the exact gradient is
\[
\frac{\partial \mathcal{C}}{\partial \theta_i}
=
\delta^{\lambda+1}_i \left(-\sin(\theta_i)\zeta^\lambda_i+\cos(\theta_i)\zeta^\lambda_{i+1}\right)
+
\delta^{\lambda+1}_{i+1} \left(-\cos(\theta_i)\zeta^\lambda_i-\sin(\theta_i)\zeta^\lambda_{i+1}\right).
\]
The update is ordinary gradient descent,
\[
\theta_i^\ell \gets \theta_i^\ell - \lambda \frac{\partial \mathcal{C}}{\partial \theta_i^\ell}.
\]
This yields a classical training algorithm, called QPC for Quantum Pyramidal Circuit, with \(O(n^2)\) complexity for the relevant layer-size scaling, compared with \(O(n^3)\) for previously used SVD-based or Stiefel-manifold orthogonal-network training methods [2106.07198][2212.07389].

A persistent misconception is that QOrthoNNs in these works are trained by parameter-shift on quantum hardware. The medical-image papers explicitly emphasize the opposite point: training can always be performed classically, and the main novelty lies in the exact orthogonal parameterization and its efficient angle-space optimization rather than in hardware gradient estimation [2109.01831][2212.07389].

## 4. Medical-image benchmarks and empirical behavior

The first substantial application domain for QOrthoNNs was medical image classification on MedMNIST. Two datasets were used: PneumoniaMNIST, a pediatric chest X-ray binary task with 4708 training images and 624 test images, and RetinaMNIST, originally a 5-level diabetic retinopathy task converted into a binary classification problem with 1080 training images and 400 test images. All images are \(28\times 28\), but for QOrthoNN the inputs were reduced by PCA to 4 or 8 dimensions because of hardware limits [2109.01831][2212.07389].

The reported QOrthoNN architectures were \([4,2]\) and \([8,2]\): a single orthogonal layer mapping directly to two outputs, with no hidden layer and a sigmoid activation for binary classification. The comparison involved three modes: SVB, a classical orthogonal neural network trained with singular value bounding; QPC-SIM, the angle-parameterized orthogonal network evaluated in simulation; and QPC-QHW, the same QPC-trained model executed for inference on IBM hardware. Evaluation used AUC and ACC. For non-hardware settings, experiments were repeated 10 times, with standard deviation typically \(\pm 0.01\) and sometimes up to \(\pm 0.04\); hardware experiments were run once per configuration and took from about 45 minutes to several hours, while simulator or classical training took seconds [2109.01831][2212.07389].

On PneumoniaMNIST, the \([4,2]\) configuration produced test AUC and ACC of \(0.90, 0.80\) for SVB/SVB, \(0.90, 0.80\) for QPC-SIM/QPC-SIM, and \(0.89, 0.76\) for QPC-SIM/QPC-QHW. For \([8,2]\), the corresponding test results were \(0.88, 0.81\), \(0.88, 0.80\), and \(0.89, 0.83\). On RetinaMNIST, the \([4,2]\) configuration yielded \(0.75, 0.71\) for SVB/SVB, \(0.74, 0.71\) for QPC-SIM/QPC-SIM, and \(0.73, 0.68\) for QPC-SIM/QPC-QHW; the \([8,2]\) configuration yielded \(0.83, 0.80\), \(0.84, 0.79\), and \(0.77, 0.74\) respectively [2109.01831][2212.07389].

These benchmarks support several narrow conclusions. First, simulation performance of the QPC parameterization is comparable to classical orthogonal baselines on the small tasks tested. Second, hardware inference is feasible for very small orthogonal models, and on PneumoniaMNIST the simulator-to-hardware degradation can be modest, particularly for \([8,2]\). Third, the harder Retina task is more sensitive to hardware noise. Fourth, the paper’s other quantum method, the quantum-assisted dense network, generally attained somewhat higher accuracies than the orthogonal model on these benchmarks, so QOrthoNNs were not presented as the strongest empirical classifier in that study [2109.01831].

The most striking training result appears outside the smallest hardware settings. On RetinaMNIST with a larger simulated orthogonal network \([32,16,2]\), the paper reports test accuracy of \(58.25\%\) for the SVB-based algorithm and \(75.25\%\) for the simulated quantum pyramid circuit algorithm. The reported interpretation is that optimization in the circuit-angle landscape can produce substantially different models from direct orthogonal-weight optimization [2109.01831][2212.07389].

## 5. Extensions to scientific machine learning and Bayesian anomaly detection

A major later specialization is the Quantum Orthogonal Separable Physics-Informed Neural Network, or QO-SPINN, which embeds quantum orthogonal MLPs inside SPINNs for PDE solving. In this architecture, each separable subnet is a quantum orthogonal network whose hidden linear maps are implemented by Hamming-weight-preserving circuits built from \(RBS\) gates, so the effective hidden-layer matrices satisfy \(W \in SO(d)\). The paper attributes a forward-layer complexity of \(\mathcal O(d\log d/\epsilon^2)\) to the quantum matrix-vector multiplication primitive, compared with the classical \(\mathcal O(d^2)\), with the caveat that the numerical results are entirely classical simulations of the quantum circuits [2511.12613].

The orthogonality in QO-SPINN is used not only for representation but also for stability and uncertainty quantification. The paper develops Lipschitz bounds for SPINN and for the orthogonal specialization,
\[
L_u \le r \cdot \left( \prod_{i=1}^{d} M_i \right) \sqrt{ \sum_{k=1}^{d} \left( \frac{\|\mathbf{W}_k\|_2}{M_k} \right)^2 },
\]
and exploits the exact spectral norm 1 of orthogonal hidden layers to adapt spectral-normalized Gaussian-process ideas without an explicit spectral-normalization step. Empirically, QO-SPINN outperformed SPINN and PINN on the 2D and 3D advection-diffusion tasks, performed comparably to SPINN but worse than PINN on 1D Burgers, recovered the Sine-Gordon inverse-problem parameter \(\beta=0.252\) versus \(0.253\) for SPINN when the target was \(0.25\), and showed mixed but often favorable uncertainty-calibration behavior relative to Monte Carlo dropout on Burgers [2511.12613].

A separate extension is the Bayesian Quantum Orthogonal Neural Network for anomaly detection in 3D additive-manufacturing data. This work combines Bayesian learning with orthogonal quantum-circuit parameterizations, treating the trainable circuit angles as random variables with a mean-field Gaussian posterior and optimizing the ELBO
\[
\mathcal{L}_{\boldsymbol{\theta}}(\boldsymbol{\gamma})
:=
\mathbb{E}_{q_{\boldsymbol{\gamma}}(\boldsymbol{\theta})} \left[ \log p(\mathcal{D}| \boldsymbol{\theta}) \right]
-
\frac{1}{2} \sum_{i=1}^N \left[ \Sigma_i^2 + \mu_i^2 - \log(\Sigma_i^2) - 1 \right].
\]
The proposed models include a feedforward autoencoder with orthogonal quantum-inspired bottleneck layers and a 3D orthogonal convolutional architecture, OrthoConv3D, in which flattened \(d\times d\times d\) filters form a matrix \(F^\ell \in \mathbb{R}^{k\times d^3}\) with mutual orthogonality across filters [2504.18103].

In the feedforward family, Bayesian QFNN attained the best calibration, with ECE \(0.217\), compared with \(0.257\) for FNN point estimate, \(0.221\) for FNN Bayesian, and \(0.251\) for QFNN point estimate. In the 3D family, Bayesian learning again improved calibration, but orthogonality did not improve the main architecture: 3D-CNN Bayesian achieved ECE \(0.209\), better than 3D-QCNN Bayesian at \(0.224\). The paper therefore presents a nuanced conclusion: Bayesian learning consistently improves calibration, while orthogonality helps more clearly in the simpler feedforward setting than in the tested 3D convolutional architecture [2504.18103].

The same anomaly-detection study also reports hardware experiments on IBM’s 127-qubit Brisbane device using 8-qubit orthogonal circuits. Real-hardware fidelity was lower than simulation but remained substantial: for random inputs, the average fidelity was about \(0.9545\) at \(1\text{k}\) shots and \(0.9570\) at \(10\text{k}\); for real voxel data, it was about \(0.9598\) at \(1\text{k}\) and \(0.9793\) at \(10\text{k}\). Unary post-selection and readout-error mitigation were used, and although direct orthogonal-layer MSE rose with the fraction of circuits run on hardware, the full reconstructed-output deviation in the hybrid autoencoder pipeline remained around \(1\times 10^{-8}\) when all 256 component circuits were executed on hardware [2504.18103].

## 6. Limitations, misconceptions, and open problems

Several limitations recur across the QOrthoNN literature. The first is tomography and readout overhead: because nonlinearities are classical, a multilayer QOrthoNN must estimate the output vector of each quantum orthogonal layer before proceeding to the next, which yields \(\widetilde{O}(n/\delta^2)\) cost with sign retrieval in the unary constructions. The second is qubit usage: unary encoding requires one qubit per feature, and the sign-recovery construction adds an extra qubit, so the medical-image demonstrations were restricted to 5-qubit and 9-qubit settings for 4- and 8-dimensional PCA inputs [2109.01831][2212.07389].

A second common misconception is that these models already demonstrate quantum computational advantage. The unary pyramidal circuits can be simulated classically in \(O(n^2)\), and the foundational papers are explicit that the principal gain is an exact orthogonal parameterization with \(O(n^2)\) training updates, not an exponential speedup. The later PDE paper makes a quantum complexity claim for the layer primitive, but it also states that the numerical results are classical simulations and that practical crossover depends on high-fidelity hardware, state preparation, and tomography cost [2106.07198][2212.07389][2511.12613].

Hardware limitations are equally prominent. The medical-imaging work reports instability over time, sensitivity to recalibration, dependence on the choice of qubit subset, and increasing fragility at 9 qubits. The anomaly-detection study shows that small orthogonal blocks can run with reasonable fidelity on current hardware, but only at small scale and within a larger hybrid pipeline. These results therefore support hardware compatibility rather than large-scale end-to-end quantum deployment [2109.01831][2504.18103].

Orthogonality itself is also not uniformly beneficial. The medical-image results show competitiveness rather than dominance over classical orthogonal baselines, and the anomaly-detection study explicitly concludes that in the tested 3D convolutional case orthogonality does not help. In QO-SPINNs, the paper notes a possible expressivity-versus-regularity tradeoff, including reduced ability to capture high-frequency modes, linking this to spectral bias and constrained Lipschitz behavior. This suggests that exact orthogonality functions as a strong inductive bias whose usefulness is domain- and architecture-dependent [2504.18103][2511.12613].

Open problems identified across the papers include richer Bayesian posteriors beyond mean-field Gaussian, scaling beyond unary or Hamming-weight-1 encodings, broader use of higher-Hamming-weight compound layers, systematic comparison of circuit layouts such as pyramid and butterfly, and determining when angle-space optimization yields genuinely superior minima relative to other orthogonal parameterizations. A plausible implication is that the lasting importance of QOrthoNNs may lie less in immediate hardware acceleration than in the combination of exact orthogonality, circuit-native parameterization, and hybrid training methods that transfer between quantum-compatible and purely classical implementations [2106.07198][2504.18103].

Source: https://www.emergentmind.com/topics/quantum-orthogonal-neural-networks-qorthonns