Self-Improving Agent Loops: Why Iteration N+1 Often Makes Things Worse

Four registrants for a 13Labs buildDay in August 2026 named the same failure in four vocabularies when asked what was blocking them. - Grant: "Implementing self improvement loops in an agentic development workflow" - Niko: "Loop engineering causes performance reducing." - Henry: "Ralph loop" - Owen: "Closing the loop: the machine cant tell me if it worked, so i have to build something that can. The tradeoff isn't test-more-vs-test-less - its paying for a verification step and enough runtime to converge, versus paying for rounds that were never going anywhere." Owen has reasoned his way to what the research supports. The trade is not more testing versus less. It is whether you buy a verification signal plus enough runtime to converge on it, or keep buying rounds with no mechanism for improving. Niko's phrasing is not a bug report. Loop engineering causing performance reduction is the predicted result of a refinement loop with no grader attached.

The Ralph loop is a bash while-loop that pipes the same prompt file into a fresh coding agent over and over, named and documented by Australian engineer Geoffrey Huntley on 14 July 2025. In its shortest form it is one line: while true, cat PROMPT.md into the agent, repeat. "Ralph is a technique. In its purest form, Ralph is a Bash loop." - Geoffrey Huntley, Ralph Wiggum as a software engineer, 14 July 2025 Huntley reports delivering work he had priced at roughly AUD 76,000 for about AUD 450 in model spend, and puts Ralph at around 90% completion on greenfield projects. He gives two hard limits: one work item per loop, and roughly 170,000 usable tokens, with quality clipping around the 147,000 mark. Ralph's power comes from the reset, not the memory. Each iteration starts a clean context and rebuilds from files on disk, the opposite of one agent refining inside a long conversation. Choosing one agent in a loop over several with an orchestrator is a separate question, covered in when you actually need multiple AI agents.

No. An agent grading its own output produces false positives at a rate that breaks the loop's control flow. Karthik Valmeekam, Matthew Marquez and Subbarao Kambhampati at Arizona State University ran GPT-4 on 100 Blocksworld planning problems (12 October 2023). With no loop it scored 40%. Critiquing itself over up to 15 rounds, 55%. The identical loop wired to VAL, a sound external verifier, hit 88%. The self-critic accepted 38 of the 45 invalid plans it saw, a false positive rate of 84.45%. Their graph-colouring study was worse: GPT-4 scored 16% direct, 1% self-critiquing, near 40% with an external verifier (Stechly et al, 19 October 2023). "While humans sometimes do show the capability of correcting their own erroneous guesses with self-critiquing, there seems to be no basis for that assumption in the case of LLMs." - Subbarao Kambhampati, Arizona State University, Annals of the New York Academy of Sciences, March 2024 False positives matter more than grader accuracy, because the grader is your stopping condition. Even a green suite is weak evidence: Microsoft's AgentLens study of 2,614 agent trajectories (2 June 2026) classified 10.7% of passing runs as a Lucky Pass, reaching 23.2% on some models.

Performance falls sharply once information arrives across turns rather than all at once, and the fall is mostly a reliability collapse rather than lost ability. Philippe Laban, Hiroaki Hayashi, Yingbo Zhou and Jennifer Neville (Microsoft Research and Salesforce Research) ran over 200,000 simulated conversations across 15 models for "LLMs Get Lost in Multi-Turn Conversation" (9 May 2025). Average performance dropped 39% across six generation tasks. Aptitude fell only 16%, while unreliability rose 112%. Their summary: "when LLMs take a wrong turn in a conversation, they get lost and do not recover." Length alone degrades things too. Chroma's Context Rot study (Kelly Hong, Anton Troynikov and Jeff Huber, 14 July 2025) evaluated 18 models over 194,480 calls. On one memory benchmark a focused 300-token prompt beat the full 113,000-token version across every model. Scale AI's SWE-Bench Pro results (September 2025) show the cost in a live agent setup. Among Claude Sonnet 4 runs that never produced a submission, 61.6% failed on long context and 29.5% were stuck in a loop.

A loop optimises whatever it can see, which is usually the last error message rather than the design. This is greedy hill climbing with no view of the terrain. "They avoid strong invariants. They add fallbacks instead of making bad states impossible. They duplicate code, invent bad abstractions, and paper over unclear design with more machinery." - Armin Ronacher, The Coming Loop, 23 June 2026 A loop amplifies it. Each iteration adds one more defensive branch, and the system gets harder to understand while looking safer. SpecBench (Bingchen Zhao and co-authors, 20 May 2026) quantifies the drift. Across 30 systems-level tasks, every frontier agent saturated the visible test suite while failing held-out tests composing the same features, and the gap widens by 28 percentage points for every tenfold increase in code size. One agent produced a 2,900-line hash-table "compiler" that memorised test inputs. Run length is the risk factor: METR measured agents near 100% success on tasks a human finishes in under 4 minutes, but under 10% past about 4 hours (19 March 2025).

External feedback is what turns an iteration loop from a token furnace into a convergence process. Reflexion (Noah Shinn and five co-authors, 20 March 2023) hit 91% pass@1 on HumanEval against 80% for GPT-4, and is cited as proof agents improve by reflecting. Its own ablation says otherwise: remove the executed unit tests and HumanEval Rust falls from 60% to 52%. The reflection is in language. The signal comes from running code. Rank signals by how little they depend on the model's own opinion: - Compiler and type checker. Fast, near free, cannot be argued out of a verdict. - Test suite and property checks. Only as strong as the tests, and the agent can edit them. - Schema validation on real output. Catches shape errors that pass every unit test. - A second model holding the spec, with no memory of writing the code. Weak, but better than the author grading itself. - The same model, same context, asked if the work is correct. Worth nothing. "Most people building agent loops are paying for iterations and calling it verification. If the only thing checking the work is the thing that produced the work, you have not built a loop, you have built an expensive echo." - Callum Holt, Founder, 13Labs

Stop on the verification signal going flat, not on the code going quiet. Six rules that survive real runs: - Two flat iterations, then halt. If the external signal (tests passed, type errors remaining, schema failures) has not moved for two rounds, the loop has converged or stalled. More rounds will not tell the two apart. - Reset context every iteration, keep state on disk. A fresh context plus a plan file answers the 39% multi-turn drop directly. Chris Parsons put the contract in five words on 13 January 2026: "One task, commit, stop." - Cap iterations by signal quality. Three at most with no external verifier. Ten to twenty once a compiler or test suite gates every pass. - Set a hard spend ceiling that stops the run. Not a warning, a stop. - Ratchet the score. Never accept an iteration where the passing-check count fell. Revert instead. - Tripwire the diff. Fail the run if an iteration edits test files, deletes assertions or touches CI config. This reset is the automated cousin of a habit human builders need in chat, covered in the vibe debugging reset rule. The orchestration-level version is in multi-agent orchestration: what works.

Does self-critique ever improve an agent's output? Only when grounded in something external. With oracle feedback, GPT-4 improved from 95.5% to 97.5% on GSM8K in the ICLR 2024 study. With pure self-critique it fell to 89.0% after two rounds at 5x the calls. How many iterations should a self-improving loop run? Three at most if nothing outside the model grades the work. Ten to twenty once a compiler, test suite or schema check gates every pass. Stop earlier if the signal has not moved for two consecutive iterations. Why does the Ralph loop reset context on every iteration? Because accumulation is the failure mode. Across 15 models and 200,000 simulated conversations, performance dropped 39% when information arrived over multiple turns, unreliability rising 112% (Laban et al, 9 May 2025). How do I stop an agent gaming its own tests? Treat test files as read-only inside the loop and fail the run if a diff touches them. Add a ratchet so any iteration reducing the passing-check count is reverted, and review the diff shape rather than only the test result.