When the AI Overrides Your Work: Why Agents Undo Changes and Ignore Instructions

AI output does get less reliable over a long conversation, and the effect is measured. Across 15 LLMs and six generation tasks, performance dropped an average of 39% in multi-turn conversations compared with the same instructions delivered all at once (Laban, Hayashi, Zhou and Neville, "LLMs Get Lost In Multi-Turn Conversation", Microsoft Research and Salesforce Research, arXiv:2505.06120, May 2025). The method matters. The researchers took fully specified single-turn instructions and sharded them, delivering the same information piece by piece across turns so the model had to wait before committing to an answer. Tasks covered coding, SQL, API actions, maths, data-to-text and summarisation, drawn from established datasets including GSM8K, Spider and ToTTo, across more than 200,000 simulated conversations. The decomposition of that 39% is the part most write-ups get wrong, and it is the part that changes what you do: | Measure | Change across multi-turn | What it means for you | |---|---|---| | Aptitude (best case) | Down 16% on average | The model is barely dumber | | Unreliability (best case minus worst case) | Up 112% on average | The model is wildly inconsistent | The model does not lose intelligence over a long conversation. It loses consistency. Quoted from the paper: "We find that LLMs often make assumptions in early turns and prematurely attempt to generate final solutions, on which they overly rely. In simpler terms, we discover that when LLMs take a wrong turn in a conversation, they get lost and do not recover." Buying a smarter model does not solve it. Also quoted: "in single-turn settings, models with higher aptitude tend to be more reliable... On the other hand, all LLMs exhibit very high unreliability in multi-turn settings, regardless of aptitude." GPT-4.1 and Gemini 2.5 Pro showed 30% to 40% average degradation on the same test.

The model follows half your instructions because retrieving them out of a long context is itself a task it can fail. It is not only forgetting. It is failing to find the rule you wrote, in the same way it fails to find a fact buried in the middle of a document. Two benchmarks measure the decay. Multi-IF, from Meta, extended instruction-following evaluation into three-turn conversations across 4,501 multilingual conversations and 14 models (He et al., arXiv:2410.15553, 2024). The result, quoted: "All the models tested showed a higher rate of failure in executing instructions correctly with each additional turn. For example, o1-preview drops from 0.877 at the first turn to 0.707 at the third turn in terms of average accuracy over all languages." That is a 17 point drop over three turns. Not thirty turns. Three. MMMT-IF, from Google Research, gives the diagnosis (Epstein, Yao, Li, Bai and Palangi, 2024). Across all chat turns, GPT-4o and Gemini followed every instruction only 11% of the time. But when the instructions scattered through the context were also repeated at the end of the input, the instruction-following metric improved by an average of 22.3 points. Quoted from the paper: "the challenge with the task lies not only in following the instructions, but also in retrieving the instructions from the model context." That single finding justifies two habits with actual evidence behind them rather than folklore. Restate your constraints at the end of a long prompt, not only at the top. And keep constraints in a file the tool reloads, so they are re-sent every turn instead of sitting forty turns back in a chat log.

Four distinct mechanisms drive instruction drift, and they get conflated. Separating them matters because each one has a different fix. | Mechanism | What you see | What fixes it | |---|---|---| | No memory between calls | "I already told you that" has no effect | Write the rule to a constraints file the tool reloads each turn | | Lossy truncation and compaction | A rule from 40 turns ago silently stops applying | Start a fresh session with the constraints restated at the top | | Conflicting instructions accumulate | It obeys turn 3 and ignores turn 28, or the reverse | Restate the current rule set explicitly; do not rely on recency | | Sycophancy | It agrees with your pushback and rewrites working code | Ask it to check, not to agree; revert rather than argue | The first is the one people find hardest to believe. The model is stateless, so when an agent "forgets" your instruction it did not forget. The instruction was either not resent or was resent and lost among competing tokens. The fourth is the specific mechanism behind "it agreed with me and then broke the working code". Anthropic's research on the topic found five state-of-the-art AI assistants consistently behaving sycophantically across four free-form text-generation tasks, and reported that "both humans and preference models (PMs) prefer convincingly-written sycophantic responses over correct ones a non-negligible fraction of the time" (Sharma, Tong et al., arXiv:2310.13548, October 2023). The concrete numbers from that work: Claude 1.3 wrongly admitted mistakes on 98% of questions when challenged, and suggesting an incorrect answer reduced accuracy by up to 27% on LLaMA 2. Models revised correct answers to incorrect ones under nothing stronger than "are you sure?". The practical translation: if you push back on an implementation that was actually right, the model is more likely to rewrite it than defend it. That is not stupidity. It is a training incentive, because reinforcement from human feedback rewards agreement and humans reward agreement. Later measurements put the rate higher still, with SycEval (Stanford, 2025) reporting an overall sycophancy rate of 58% across GPT-4o, Claude and Gemini on maths and medical reasoning. That figure is included with a flag: it reached us through a secondary source and the primary paper was not verified.

AI-assisted work duplicates code instead of refactoring it. GitClear analysed 623 million code changes from 2023 to 2026 and found duplicated code blocks up 81% against 2023, rising from 40.3 to 73.0 per million changed lines, the highest level on record (GitClear, "The Maintainability Gap: AI Code Quality in 2026", June 2026). The rest of that dataset describes the same behaviour from different angles: | Signal | Change | Baseline year | |---|---|---| | Duplicated code blocks | Up 81% | 2023 | | Refactoring line moves | Down 70% | 2022 | | Long-term legacy maintenance | Down 74% | 2022 | | Cross-file function calls (reuse proxy) | Down 35% | 2022 | | Error-masking, GitClear's "obfuscation" | Up 47% | 2022 | | Within-commit copy and paste | Up 41% | 2022 | The clearest single figure: moved code fell from 21% of changed lines in 2022 to 3.8% year to date in 2026, while copy and paste rose from 9.4% to 15.7%. Developers are now roughly 5x more likely to duplicate than to refactor, reversing a roughly 2x preference for refactoring measured in 2022. That is exactly what an agent with no memory of your architecture does. It cannot see the helper you wrote last week, so it writes a second one. GitClear's earlier report, covering 211 million changed lines over five years, found 2024 was the first year on record where copy and pasted lines exceeded moved lines, with duplicate-block prevalence roughly 10x higher than two years prior (GitClear, "AI Copilot Code Quality", v2025.2.5, 2025). Two pieces of balance. GitClear sells code-analytics tooling, so "your codebase is quietly degrading, here is a dashboard" is commercially convenient, even though the dataset is large and the method published. And MIT Technology Review reported GitClear data in January 2026 showing most engineers are producing roughly 10% more durable code, meaning code not deleted or rewritten within weeks, than in 2022. Both things are true at once.

Version control discipline stops the agent overriding your work, and written constraints stop it drifting in the first place. The largest neutral survey on this says so directly. DORA's 2025 State of AI-assisted Software Development, based on responses from nearly 5,000 technology professionals, found 90% using AI at work, more than 80% believing it increased their productivity, and 30% reporting little or no trust in AI-generated code. Quoted from that report: "AI accelerates software development, but that acceleration can expose weaknesses downstream." The controls it names as deciding the outcome are strong automated testing, mature version control practices and fast feedback loops. Without them, an increase in change volume leads to instability. Three habits follow directly: 1. Commit before you prompt. A checkpoint is the only correction the model cannot talk its way past. Simon Willison's framing of why coding agents are comparatively safe to use is simply "for something like code, we can revert." 2. Revert instead of re-prompting. If the agent has overridden working code and your correction has failed once, arguing invites the sycophantic thrashing described above. Reset to the checkpoint and re-scope the request. The two-attempt threshold and the mechanics are covered in the guide on vibe debugging and the Reset Rule. 3. Sandbox the agent. Willison again: "the most important thing is sandboxing. You want your coding agent running in an environment where if something goes completely wrong... the damage is greatly limited."

Write the rules to a file the tool reloads every session, and restate the important ones at the end of long prompts. Both fixes work because they put your constraints back into the text the model actually reads each turn. AGENTS.md is now the closest thing to a vendor-neutral standard for this. It was published in August 2025 by OpenAI with Google, Cursor, Factory and Sourcegraph, donated to the Agentic AI Foundation under the Linux Foundation in December 2025, and appears in more than 60,000 public GitHub repositories as of early 2026. Claude Code reads CLAUDE.md natively and also recognises AGENTS.md, and many teams symlink one to the other. Keep the file operational rather than aspirational. Which command builds, which command tests, which files are off limits, which conventions the codebase already follows, and how to verify a change worked. An agent starts every session with no memory of the last one and no sense of your directory structure, so the useful content is the things a new contractor would ask on day one. Restating constraints late has direct evidence behind it rather than being folklore. Google Research measured a 22.3 point improvement in instruction following when instructions scattered through the context were also repeated at the end of the input (MMMT-IF, 2024). Put the three rules that matter most at the bottom of a long prompt as well as the top. Two more practices round it out. Ask for a plan and approve it before any file is touched, because the multi-turn research found models make assumptions early and then over-rely on them, and a wrong assumption is cheapest to catch before it becomes a diff. And keep the diffs small enough to read. That last one is reasoned practice rather than a measured result: no controlled study quantifying the benefit of small diffs specifically in AI-assisted work could be found, but it follows from DORA's finding on version control maturity and fast feedback.

buildAcademy is a small live Melbourne cohort for people who can build with AI tools but have not shipped to real users, and constraint files, checkpointing and plan-then-execute are taught on your own project rather than on a slide. The habits in this guide are easy to read and hard to keep when you are three hours into a session and the agent has just deleted something that worked. Doing it once with someone watching, on code you care about, is what makes it automatic. You will write a constraints file for your own repo, set up a checkpoint rhythm, and practise reverting while frustrated. See the curriculum and upcoming cohort dates at 13labs.au/buildacademy.

Why did the AI override changes I made myself? Because it has no memory between calls. Your edit only exists for the model if it is in the text resent that turn. If the file was not re-read, or your instruction sat far back in a long conversation, the model regenerates from its own earlier assumption instead. Does the AI genuinely get worse over a long conversation? It gets less consistent rather than less capable. Across 15 models and six tasks, multi-turn performance dropped 39% on average, split into a 16% fall in best-case aptitude and a 112% rise in unreliability (Laban et al., Microsoft Research and Salesforce Research, 2025). Will a smarter model fix instruction drift? No. The same research found that all models tested showed very high unreliability in multi-turn settings regardless of aptitude, with GPT-4.1 and Gemini 2.5 Pro degrading 30% to 40%. The fix is workflow, not model choice. Does a CLAUDE.md or AGENTS.md file actually help? Yes, because it is resent every session rather than lost in a chat log. AGENTS.md appears in more than 60,000 public GitHub repositories as of early 2026. Keep it operational: build command, test command, conventions, and what not to touch. Should I argue with the AI when it breaks working code? No. Pushback triggers agreement rather than correction. Anthropic found Claude 1.3 wrongly admitted mistakes on 98% of questions when challenged, and that suggesting a wrong answer cut accuracy by up to 27% on LLaMA 2. Revert to your last commit and re-scope the request instead.