Jointly learned encoders with theoretical guarantees

Establish whether jointly learned Transformer encoders can achieve the same memory and accuracy guarantees as the explicitly constructed encoders for fixed-policy TD policy evaluation.

Background

The paper proves optimal or near-optimal context and memory guarantees for explicitly constructed encoders, including Krylov and sample-and-hold constructions. It also studies learned bottleneck encoders empirically, finding that standard Transformer bottlenecks do not reliably discover the structures needed for accuracy to improve with context size. The theoretical learnability result concerns only the shared executor, not the encoder, leaving joint encoder learning unresolved.

References

Whether jointly learned encoders can achieve the same memory and accuracy guarantees and whether these results extend to policy control, remain open questions.

— Provable Transformer Context Compression for In-Context Reinforcement Learning  (2610.20004 - Zhao et al., 14 Oct 2026) in Conclusion; Section I, “Scope and Limitations”

whether a uniformly computable time-invariant encoder attains Θ(Ψ_m) remains open.

— Provable Transformer Context Compression for In-Context Reinforcement Learning  (2610.20004 - Zhao et al., 14 Oct 2026) in Appendix D, Section “Statements,” paragraph following Proposition D.1; Appendix D, subsection “Numerically Optimized Time-Invariant Recurrences”