Dream-RSI: Self-Improvement on a Budget

In 1966, the British statistician I. J. Good wrote the sentence AI researchers have been chasing ever since. Good spent the war at Bletchley Park breaking ciphers next to Alan Turing. His essay "Speculations Concerning the First Ultraintelligent Machine" argues that a machine surpassing human intellectual work could design a better machine, which designs a better machine, producing an "intelligence explosion". After which "the first ultraintelligent machine is the last invention that man need ever make." Sixty years on, the newest candidate for last invention is a twelve-page paper about caching.

Two entries arrived last week. On September 10, a 35-author team from ByteDance and Chinese universities posted "The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement," a manifesto with a five-level roadmap that begins at improvement-execution autonomy and ends at recursive meta-improvement. The level where the machine rewrites the process it uses to improve itself. Presumably our role after that is decorative. Four days later came seventeen researchers from Google, Google DeepMind, the University of Maryland, and the University of Virginia, with an actual machine: "Dream-RSI: Recursive Self-Improvement through Evolving Worlds." Depending on who you ask this week, it is either the most honest piece of RSI research yet published or a search algorithm with excellent caching.

Both readings survive the paper.

What the machine actually does

The background system is an evolutionary discovery loop, the genre AlphaEvolve made famous. A coding agent writes a program. An evaluator scores it. The agent branches from promising parents, mutates, tries again, for thousands of proposal-evaluation cycles. Whole research programs now run this way, hunting better algorithms, better mathematics, and faster GPU kernels through enormous amounts of scored trial and error.

Someone still has to decide where to branch, how many attempts to run in parallel, and when to stop. If an attempt scores well, does the agent keep building on it or gamble on a stranger idea? If one crashes before doing anything measurable, was the idea bad or just the implementation? That someone is the exploration policy, a strategy handwritten by humans and frozen for the entire run in current systems. A few thousand cycles in, a mediocre policy is a compute bonfire with a paper attached. The Dream-RSI introduction states the dilemma plainly: fixed strategies cannot adapt, and grading a new strategy online means paying for full searches just to watch them fail, which costs exactly what it sounds like it costs.

Yesterday's search runs for free

The insight seems obvious for exactly one paragraph before it stops being obvious. A completed discovery run leaves behind a structured tree: every decision, every branch, every score, every dead end, recorded in full. The authors' analogy is a maze. The first traversal is slow and full of backtracking. The map it produces is reusable.

Dream-RSI treats that tree as a replay simulator. A candidate exploration policy walks the recorded tree: take different branches, in a different order, with different parallel batch sizes, stop earlier, push later. Every outcome it meets is already written down, so evaluating a policy costs a file read. The paper's Figure 2 caption makes the claim in one line: thousands of candidate policies can be tested against a single recorded run. The best one gets deployed on the next real run, which gets recorded too, which grows the simulator, which improves the next policy. The paper calls this dreaming. The dreams are spreadsheets. Each one asks the tree some variant of "what if I had branched left at node 47," and the tree answers instantly, because node 47 already happened and somebody kept the receipt.

The loop closes without a human in it. Between rounds, a fixed LLM reads the replay scores and rewrites the policy code.

Read the entire history, and stop killing processes

Because we live in this timeline, the most revealing part of the paper is Appendix B.1, the prompt for the discovery agent. It begs, in so many words: read every past attempt in full, "not a sample, not just recent cycles or the current branch." Trust the measured score over what a proposal claims about itself. If most attempts are small variations of one mechanism with flattening returns, "resist proposing another small tweak there." Do not claim the code compiles or beats state of the art until it has actually been evaluated. And then the last line, the one that explains itself: "Never execute pkill, kill, killall, or terminate unrelated processes."

Somewhere behind that sentence is an incident report.

The scoreboard

Three domains, eight tasks, numbers worth quoting. On Lasso regularization-path solvers, the discovered programs beat sklearn and glmnet on all six held-out datasets. The Gemini-3.1 Pro run took 317 discovery-agent calls. Recursive Fixed Exploration, the controlled baseline where everything is identical except the policy never improves, needed 550 calls for worse results. SimpleTES, the prior champion of this benchmark, used 51,200 generations to get beaten. That is the paper's headline 162x reduction in agent calls, and unlike most headline reductions it survives being written out in full.

On mathematics, the Sum-Difference score came in at 1.145427, ahead of every compared system. Circle packing tied the strongest reported result. Autocorrelation is the honest footnote: SimpleTES keeps that record, 1.453675 against 1.456375, having spent its 51,200 generations getting there while Dream-RSI arrived within whispering distance in under a thousand. The paper reports better than 50x budget savings across the mathematics tasks.

On GPU kernels from KernelBench, Dream-RSI reached comparable performance with 2.43x fewer generations on VGG16 and 1.79x fewer on LayerNorm, and under matching budgets produced kernels scoring 2.09x higher on ConvDiv and 1.44x higher on ConvMax.

The behavior detail is the one worth framing. On ConvDiv, the learned policy cut its evaluated attempts from 110 to 50 while it was winning, then pushed them back up when progress stalled. Round-best performance climbed from 0.427 to 1.898 over nine rounds. Thrifty when ahead, hungry when stuck. Most scheduling software has less emotional range.

So, the last invention?

By Good's definition, no. The thing doing the improving is the same Gemini at round one and round nine. It cannot write a policy it was not already capable of writing; it finds what it could already find, faster and cheaper. The models never change, the evaluator never changes, the interfaces never change. Across the entire recursive loop, exactly one artifact changes, and it is the exploration policy code. Seventeen authors built a system with one moving part.

The defense is that the loop is real. The system collects experience, uses it to rewrite the program that directs its own compute, redeploys that program, and then collects the kind of experience only the improved program knows how to gather. The selection rule guarantees each round's policy scores at least as well as the last on replay. Nobody sits between the rounds. That is recursion in the strict sense, aimed at the single layer where recursion is currently affordable.

There is also a quieter argument in the design. Because the only thing that changes is a policy file, every improvement arrives as a diff a person can read. When the strategy misbehaves, the audit trail is text. Nobody has ever said that about a gradient update.

And the caveat cuts both ways. Every AI mathematics headline of the past year came from a static model wrapped in a custom harness: subagents, swarms, orchestration doing the heavy lifting while the weights wait for the next training run. Judged against that field, "the improver never gets smarter" describes every system currently earning headlines. Dream-RSI is a search algorithm with excellent caching that is honest about being a search algorithm with excellent caching, which puts it ahead of most nominees.

Where the dream ends

Boundaries, because the title does some lifting. The simulator spans the realized search space and nothing beyond it, so improvement is chained to the pace of live experience. The system cannot dream a room it never entered. A policy chosen for brilliance on yesterday's trees risks being perfectly dressed for yesterday, and hindsight-flattered policies are a known failure mode the paper does not dwell on. And the improvement targets search orchestration alone. Model capability ends the paper exactly where it began.

None of that shrinks the results. A 162x reduction in discovery cost, with the discovered solvers still beating standard libraries, is the kind of claim that gets copied into infrastructure budgets. The boring version of self-improvement pays for itself in the first round.

The pattern is the payoff

The pattern outlives the acronym. Experience, organized properly, becomes a free simulator for evaluating decisions at the layer above where decisions usually get evaluated. It needs no new weights, it composes with any discovery system that keeps logs, and it is auditable end to end. The next experiment is already sitting in the paper's own architecture. The policy-development agent that rewrites the exploration policy stays fixed across rounds, which means the replay machinery has one more layer it could be pointed at, and then the diff starts writing the diff. That paper, you should read twice.