Attention-Like Hebbian Learning from Quantum Probability Flow and Quantum-Annealer Tests

Abstract: We propose a quantum probability-flow principle for deriving local learning rules in associative memory. A transverse field defines leakage channels from data states, and minimizing the measured survival loss gives stability-driven updates. For imaginary-time, dephased dynamics, the local leakage free energy is the log-sum-exp of energy gaps; its gradient is a softmax-weighted Hebbian rule. Real-time stability instead yields a power-law weighting. D-Wave standard- and fast-anneal tests of a one-hot attention forward map are better fitted by an effective softmax than by a Lorentzian power law.

• The paper demonstrates that quantum probability-flow yields margin-aware Hebbian updates, interpolating between classical learning rules and softmax attention mechanisms.
• The paper employs a quantum extension of minimum probability flow with local free energy gradients, validated by D-Wave annealer experiments confirming softmax predictions.
• The paper’s findings suggest hardware-native implementations for attention mechanisms, paving the way for innovative quantum-assisted memory models in AI.

Quantum-Derived Hebbian Learning and Attention: Probability Flow Analysis and Quantum Annealer Evaluation

Theoretical Framework: Quantum Probability Flow and Hebbian-Attention Interpolation

The paper formulates a framework linking Hebbian memory reinforcement and Transformer-style attention through a quantum probability-flow principle. This is achieved by defining local learning rules within an associative memory encoded by Ising spins, parametrized by couplings $J_{ij}$ and fields $h_i$. The key mechanism is a transverse field driver ($V = \Gamma \sum_i \sigma^x_i$), which induces probability leakage channels corresponding to single-spin flips.

The quantum extension of minimum probability flow (MPF) is employed: rather than computing global likelihoods, the learning objective minimizes survival loss upon perturbation, specifically how probable a measured state remains in the original data configuration after quantum evolution and projective measurement. For imaginary-time dynamics, the local probability-flow loss is shown to be a log-sum-exp over energy gaps, yielding a local free energy. The gradient of this free energy produces a softmax-weighted Hebbian update, modulating the classical correlation reinforcement by the fragility of component spins. The resulting update formally interpolates between the classical Hebbian rule (temperature $\beta \to 0$) and an attention-like softmax mechanism (finite $\beta$), with the temperature parameter controlling margin-sensitivity.

In real-time quantum dynamics, the transition probabilities governing leakage weights scale as power-law functions of the energy gaps, offering a coherent-stability regime distinct from the exponential softmax. The paper dissects these two regimes, situating them within the broader taxonomy of attention kernels—including non-softmax variants such as entmax, Gaussian, and Lorentzian power-law biases.

Physical Interpretation of Attention as Margin Repair

The stability principle underpinning the probability-flow objective naturally defines attention as selective repair. The energy gaps $\Delta E_k$ enumerate the instability margins for possible errors arising from spin flips. Under imaginary-time evolution, local instability weights $q_k$ emerge as Boltzmann softmax distributions over these channels, directing learning resources to the most fragile correlations.

The Hebbian update is thus modulated: correlations are reinforced predominantly when endpoints are locally unstable, rather than indiscriminately. This mechanism accords a statistical-physics interpretation to attention—it arises from normalized competition among quantum error channels, not from explicit architectural imposition. Gradient descent on the averaged survival loss yields parameter updates concentrated on weak barriers, facilitating targeted refinement of associative memory margins.

The formulation extends to multiple heads by decomposing the driving Hamiltonian. Each component can probe distinct error modes (single-spin vs. multi-spin flips), with individual local softmax distributions, thereby aligning "multi-head attention" with simultaneous stability testing across geometries.

Quantum Annealer Experiments: D-Wave Forward Map Evaluation

The practical viability of the theoretical constructs is tested on a D-Wave Advantage quantum annealer. The experiment encodes a one-hot attention selection map by mapping query-key dot products to scores, and imposing energy penalties to enforce valid one-hot assignments. The annealer samples from this Ising energy landscape, and final readouts are analyzed for their adherence to softmax and power-law distributions.

Empirical results reveal strong agreement with softmax forward maps: fitted slopes $\kappa/\lambda$ track the programmed score scale consistently across standard- and fast-anneal protocols. Global KL divergences are consistently smaller for the softmax family compared to power-law (Lorentzian) alternatives. Notably, fast annealing (sub-microsecond) reduces the effective softmax slope but does not enhance preference for the power-law fit, confirming that D-Wave readout remains more closely described by thermal (softmax/free-energy) attention rather than coherent response kernels.

This evidence substantiates the claim that quantum annealers natively support hardware-based Boltzmann softmax attention, as opposed to the coherent power-law weighting predicted by real-time quantum transitions.

Implications and Perspectives for AI and Quantum Hardware

The results articulate a unifying framework for memory-driven learning and attention, rooted in quantum statistical physics. The interpolation between Hebbian and attention mechanisms is obtained from underlying stability principles and dissipation, rather than algorithmic construction. This theoretically grounded approach allows margin-aware associative memory updates and suggests direct hardware implementations leveraging quantum annealers or stochastic Ising machines as samplers.

Practically, the findings indicate that quantum devices can serve as energy landscape processors for forward attention maps, potentially circumventing the need for explicit score computation and normalization in digital hardware. The broader implications include physical implementations of interacting attention—where underlying energy functions produce candidate dependencies beyond standard Transformer architectures. This may facilitate novel architectures with collective or geometrically structured error channels, and suggests a pathway for the realization of multi-head attention as parallel stability probes.

Future directions include:

• Systematic benchmarking of interacting forward maps on quantum annealers, extending beyond non-interacting baselines.
• Exploration of alternative driving Hamiltonians for richer error channel probing.
• Integration of quantum margin-aware refinement procedures into classical memory models.
• Assessment of hardware efficiency in Boltzmann attention versus coherent-stability computations.

Conclusion

This study rigorously demonstrates that both Hebbian memory reinforcement and attention-like softmax weighting arise from a quantum probability-flow stability principle, with interpolating dynamics determined by temperature and dissipation regime. Quantum annealer experiments validate the realization of softmax attention as a physical process, supporting quantum hardware-native approaches to margin-aware learning and associative memory refinement. The theoretical and empirical results imply a paradigm shift in how memory and attention can be jointly formulated and physically realized.