Dream-RSI: Recursive Self-Improvement through Evolving Worlds
This presentation explores how automated discovery systems can recursively improve their own exploration strategies by converting completed search histories into replay simulators. Dream-RSI demonstrates that replay-based off-policy evaluation enables more efficient meta-level optimization across algorithm engineering, mathematical optimization, and GPU kernel development, achieving comparable or superior solution quality while reducing discovery costs by up to 2.4×.Script
Most recursive self-improvement systems struggle with a hidden bottleneck: they can't easily experiment with different exploration strategies because testing each one requires expensive, time-consuming discovery runs. What if we could turn completed search histories into fast-forward simulators?
Dream-RSI treats every completed discovery run not as throwaway context, but as an empirical simulator. Each historical tree preserves exact workspace states, generated artifacts, and evaluation scores, allowing thousands of candidate exploration policies to replay different branch selections and scheduling decisions without re-invoking expensive code generation or evaluation.
The algorithm alternates between three phases. First, the current policy conducts one fresh online rollout, generating a new discovery tree. That tree joins the history, and an offline phase begins: a policy-development agent edits the exploration code, each revision is replayed across all historical trees, and the highest-scoring variant becomes the next deployed policy.
Across Lasso regularization engineering and GPU kernel benchmarks, Dream-RSI achieves comparable or superior solution quality while reducing discovery costs substantially. For Lasso with Gemini-3.1 Pro, it reached better runtime performance using 1.7 times fewer generation calls than the fixed baseline.
Behavioral analysis reveals something unexpected: the learned controller adapts its compute allocation nonmonotonically. Early rounds reduce evaluated attempts from 110 to 50 as performance climbs, then later increase effort again when progress plateaus, triggering renewed gains. This suggests the policy has learned when to exploit existing branches and when to invest in broader exploration.
Replay-based policy improvement offers a concrete path forward: when we preserve the structure of discovery, we can learn better ways to explore without repeatedly paying the cost of exploration itself. Visit EmergentMind.com to dive deeper into Dream-RSI and generate your own video explanations of cutting-edge research.
