Adversarial Reinforcement Learning for Adaptive Eavesdropping in BB84 Quantum Key Distribution

Abstract: BB84 quantum key distribution derives its security from a physical guarantee that any eavesdropper disturbs the channel in a statistically detectable way. Prior work evaluates this by assuming Eve attacks at a fixed, analytically optimized rate. We examine what happens when Eve is modeled instead as a learning agent. Classical reinforcement learning is used, specifically tabular Q-Learning, SARSA, and Double Q-Learning, to adaptive BB84 eavesdropping. This formulates the attacker's decision as a Markov Decision Process where the agent observes Quantum Bit Error Rate (QBER) feedback and decides, qubit by qubit, whether to intercept or pass. Experiments span three channel noise levels ($μ{ch}\in{1\%,3\%,5\%}$) and are validated across five independent random seeds (45 training runs per condition, 10,000 episodes each). Against the best non-adaptive analytical baseline, Q-Learning reduces detection from $99.4\%$ to $0.28\%\pm0.27\%$ at $μ{ch}=1\%$ while extracting approximately 10.5 correct bits per episode. This is a 355-fold reduction that is statistically significant ($p=0.020$, Mann-Whitney $U$ test). We also report the spontaneous emergence of an end-game burst, where agents independently learn to surge their attack rate at the final block. This exploit vanishes under randomized checkpoint intervals while stealth performance remains statistically indistinguishable. These results motivate the inclusion of adaptive adversary baselines in quantum cryptographic security evaluations.

• The paper shows that classical RL agents (Q-Learning, SARSA, Double Q-Learning) learn adaptive eavesdropping strategies that drastically reduce detection rates in BB84 QKD.
• It formulates the eavesdropper’s attack as an MDP using QBER feedback, exposing vulnerabilities in fixed-rate security analyses.
• The study recommends protocol defenses, such as randomizing checkpoint intervals, to mitigate emergent temporal attack patterns in practical deployments.

Adaptive Reinforcement Learning Strategies in BB84 Quantum Key Distribution Eavesdropping

Introduction

This study examines the impact of adaptive adversarial strategies in the BB84 Quantum Key Distribution (QKD) protocol, specifically the effect of learning-based eavesdroppers who dynamically modulate their attack rate in response to observed channel feedback. The classical security analysis typically assumes that Eve attacks at a fixed analytically optimized rate and ignores her ability to leverage Quantum Bit Error Rate (QBER) feedback. This paper formulates Eve's eavesdropping as a Markov Decision Process (MDP) and deploys classical tabular RL approaches (Q-Learning, SARSA, Double Q-Learning) to empirically demonstrate substantial reductions in detection rates compared to fixed-rate attacks. Several structural vulnerabilities and defensive countermeasures are analyzed, with implications for both practical BB84 deployments and future quantum cryptography protocol evaluation.

Formulation and Methodology

The attacker's policy is constructed as an MDP, with state space defined by (QBER, attack count per block, block index) and actions {pass, attack} applied per qubit. The environment incorporates realistic channel noise ($\mu_{ch}$) as well as physical BB84 error dynamics. Eve receives per-qubit and per-checkpoint rewards, heavily penalizing detection (QBER exceeding 11%) and incentivizing information extraction without detection.

Three tabular RL agents are trained: Q-Learning (off-policy, max operator), SARSA (on-policy), and Double Q-Learning (using two Q-tables to correct bias). Hyperparameters are empirically optimized for each noise regime. Baselines include "Always Attack" and the analytically optimal Fixed Rate (as in Lee et al. [11]), which provide upper bounds for information gain and stealth, respectively.

Each agent-noise configuration is validated with five independent seeds, producing statistically robust detection and information collection metrics across 45 runs per condition (10,000 episodes each).

Experimental Results

Detection and Information Leakage

At channel noise $\mu_{ch}=1\%$, Q-Learning achieves a detection rate of $0.28\% \pm 0.27\%$ compared to $99.4\%$ for Fixed Rate attackers, representing a statistically significant (Mann-Whitney U, $p=0.020$) reduction by a factor of 355. Information leakage per episode remains at $10.5$ correct bits, substantially lower than the $46.6$ bits achieved by the reckless Always Attack baseline, but undetectable by QBER thresholding. SARSA and Double Q-Learning similarly outperform baselines, though exhibit more conservative policies consistent with their update rules.

With increased noise ($\mu_{ch}=5\%$), detection rates converge ($>88\%$) across all agents, as channel-induced errors saturate the available stealth window.

Stealth-Information Trade-off

Experiments demonstrate a clear inverse relationship between stealth and information gain. RL agents prioritize evasion, attacking infrequently to maximize survival across checkpoints, with mean information leakage per session well below fixed-rate or full-attack baselines. Detection and information metrics shift predictably as channel noise increases, reflecting reduced safe attack budgets.

Emergent Temporal Attack Patterns

A critical observation is the spontaneous emergence of an "end-game burst": RL agents universally learn to escalate attack rates at the terminal block of fixed-length episodes, exploiting the absence of future detection costs. This pattern is robust across all three algorithms and seeds. Upon randomizing checkpoint intervals, the burst is eliminated, replaced by a more uniform attack distribution, without degrading overall stealth performance. This indicates that the burst is a protocol structural artifact, not a primary vulnerability.

Statistical Robustness

Stealth performance differences are statistically significant across most agent pairwise comparisons at $\mu_{ch}=1\%$ and $\mu_{ch}=1\%$0. At higher noise, where strategic latitude diminishes, inter-agent differences are insignificant. All results are reproducible across seeds, confirming genuine learning is achieved.

Implications and Recommendations

Security Margin Reduction

The findings call into question standard fixed-rate security models for BB84. Adaptive RL attackers exploiting QBER feedback dramatically compress the practical security margin, enabling persistent undetectable leakage well below QBER thresholds. For $\mu_{ch}=1\%$1, the analytical threat model underestimates attack effectiveness by orders of magnitude.

Classical RL as Analysis Tool

Classical tabular RL provides full transparency, reproducible results, and interpretable policies, making it an ideal tool for exposing structural protocol exploits. Notably, the demonstrated attack capability requires no quantum hardware and is realizable today by any adversary with access to channel taps and public QBER broadcasts. This highlights the real-world relevance of the findings.

Structural and Defensive Recommendations

Three core recommendations are derived:

1. Randomize Checkpoint Intervals: Prevents end-game burst pattern, mitigating temporal exploits.
2. Reduce QBER Thresholds at Low Noise: Narrows stealth window; however, must balance against false detections from channel fluctuations.
3. Include Adaptive Adversaries in BB84 Security Audits: Adaptive RL exploitation is sufficiently potent to warrant explicit modeling in protocol evaluations.

Limitations and Future Directions

The study assumes non-adaptive defender protocols, ideal channel optics, and discrete state-action spaces. Real-world deployments may exhibit additional noise sources, hardware artifacts, and potential for defender-side adaptation. Extending the model to continuous RL agents, hardware-in-the-loop simulations, and adversarial defender modeling is a necessary avenue for future research. Larger-scale statistical replication is also recommended.

Conclusion

This proof-of-concept demonstrates that classical RL agents—without quantum hardware—can learn to eavesdrop on BB84 in a manner that is nearly undetectable by standard QBER thresholding at low channel noise. Emergent attack patterns reveal protocol structural vulnerabilities, while stealth performance persists even under defenses that eliminate temporal exploitability. These findings necessitate reframing BB84 security evaluations around adaptive adversaries and motivate deeper investigation into the interplay between protocol structure and learning-based attacks. Standard analyses based on fixed-rate attackers are insufficient to capture the full adversarial potential in practical QKD deployments.

Reference: "Adversarial Reinforcement Learning for Adaptive Eavesdropping in BB84 Quantum Key Distribution" (2606.22962).