Krylov-type compression for in-context control computations

Determine whether the control computations implemented by existing in-context reinforcement-learning constructions, including LinUCB, Thompson sampling, UCB-VI, semi-gradient SARSA, actor–critic, and MASTER, admit a Krylov-type compression despite the relevant subspace changing with the policy.

Background

The paper’s compression theory applies to fixed-policy TD policy evaluation, where the relevant Krylov subspace is generated by a fixed empirical operator and reward vector. Other in-context reinforcement-learning constructions execute control or decision-making algorithms whose policies evolve, changing the relevant subspace. The paper explicitly leaves open whether these computations can nevertheless be compressed using a Krylov-type representation.

References

Whether the control computations they execute admit a Krylov-type compression is open, since the relevant subspace changes with the policy.

— Provable Transformer Context Compression for In-Context Reinforcement Learning  (2610.20004 - Zhao et al., 14 Oct 2026) in Appendix G, paragraph “Constructions for other in-context RL problems”