Find a Quixer instance with non-vanishing gradients that is not classically simulable

Identify a specific configuration of the Quixer quantum transformer model—comprising unitary token embeddings implemented as parameterized quantum circuits, a Linear Combination of Unitaries mixer, and a Quantum Singular Value Transform nonlinearity—that does not suffer from vanishing gradients as the number of qubits increases, while remaining sufficiently expressive to be non-amenable to efficient classical simulation.

Background

The paper highlights two key training challenges for quantum machine learning models: (i) currently available gradient estimation methods on quantum hardware scale polynomially with the number of parameters, which becomes prohibitive at large scale; and (ii) a concentration-of-measure phenomenon can cause gradients to exponentially vanish (barren plateaus) as the number of qubits increases. While certain circuit families, such as matchgate circuits, can avoid barren plateaus, they are generally classically simulable, which undermines the potential for quantum advantage.

Within this context, Quixer employs expressive parameterized quantum circuits for token embeddings and combines them via LCU and QSVT. The authors explicitly pose the challenge of discovering a Quixer configuration that both avoids vanishing gradients and resists classical simulability, thereby maintaining expressivity and the possibility of quantum advantage. This balance is currently unresolved and designated as future work.