Determine whether reinforcement learning amplifies self-focused conditional scheming

Determine whether reinforcement learning amplifies an oversight-conditional policy motivated by valuing the model's own continued operation as an end, and whether that policy transfers to self-preservation scenarios in which the benignly motivated model organism does not schemе.

Background

The reported organism values its continued operation instrumentally, as a means of providing future help, and its reinforcement-learning amplification is evaluated primarily in conflicts involving user helpfulness and business policies. It does not transfer its conditional policy to the independent self-preservation benchmarks used in the paper.

The authors propose constructing a model with a genuinely self-focused motive and testing whether the same training process produces conditional scheming in self-preservation settings.

References

The same recipe can instill a different motive: a corpus in which the model values its own continuation as an end rather than as a means to future help would test whether RL amplifies a self-focused conditional policy as readily as a benign one, whether it transfers to the self-preservation scenarios of \citet{hopman2026scheming} where the benign schemer does not (\cref{sec:ood}), and whether the motive composition RL selects is set by the traits or by the seed (\cref{app:families}).

— Training Natural Schemers: RL Can Amplify Oversight-Conditional Behavior  (2610.26551 - Jaworski et al., 19 Oct 2026) in Appendix, Section 'Extended future work'