Deep FHE-RL Architectures Beyond Shallow Networks

Integrate FHE bootstrapping into the Homomorphic Advantage Operator framework to enable reinforcement-learning architectures deeper than the current single-hidden-layer network and to support higher-degree polynomial activations under homomorphic-encryption constraints.

Background

The paper’s experiments are restricted to a single hidden-layer network with a degree-2 polynomial activation because of the multiplicative-depth limitations of the TenSEAL CKKS implementation. Although the Homomorphic Advantage Operator removes the Bellman-drift mechanism without adding nonlinear multiplicative depth, it does not by itself remove the depth limitations imposed by leveled homomorphic encryption.

The authors identify FHE bootstrapping as the required mechanism for scaling to deeper, multi-layer topologies and higher-degree polynomial activations. The computational expense of bootstrapping makes this an unresolved deployment challenge for privacy-preserving reinforcement learning.

References

Overcoming the strict multiplicative depth limits of shallow networks will require the integration of FHE bootstrapping, an open challenge that would enable scaling to deep, multi-layer topologies.

— Homomorphic Advantage Operator: Stabilizing Reinforcement Learning Under Fully Homomorphic Encryption Constraints  (2610.02074 - Nadhir et al., 1 Oct 2026) in Section 5, “Conclusion and Future Work”