By mid-2026, coding agents have moved from novelty to infrastructure. Teams are adopting them, budgeting for tooling, and watching benchmark leaderboards. And everywhere you look, someone is claiming their agent is “self-improving.”
That phrase actually describes four different mechanisms. Confusing them means adopting the wrong architecture, trusting the wrong benchmarks, and expecting improvements that cannot arrive from the layer you have invested in. By the end of this article you will know which layer someone means when they say “self-improving coding agent,” and why that distinction changes how you evaluate the claim — a question that sits within the broader landscape of self-improving coding agents, from architecture through to security.
What Is a Self-Improving Coding Agent and How Does It Work?
A self-improving coding agent is any AI coding system that gets better at generating, debugging, or iterating on code over time. The how differs across four layers of the stack.
Layer 1: Task-loop automation. The agent generates code, runs it, reads errors or test failures, and tries again. Each attempt learns from the previous one within a single session. This is what most practitioners mean by “self-improving.” Claude Code, Codex CLI, and Cursor all implement some form of it. It is the most common mechanism and the one you will encounter most often.
Layer 2: Skills iteration. The agent accumulates reusable methodologies across sessions and projects. SKILL.md files, prompt templates, learned heuristics. Improvements persist as composable knowledge artefacts rather than as ephemeral context. Addy Osmani describes this as the skills-as-playbooks pattern 2026 trends.
Layer 3: Harness telemetry. The agent harness records every tool call, output, and error across sessions. Engineers analyse those traces to improve the harness itself: better tool selection logic, smarter retry strategies, more precise prompt templates. Arize AI‘s Phoenix observability platform exemplifies this approach closing the loop.
Layer 4: Model-level reinforcement learning. The underlying LLM weights are updated via fine-tuning on agent trajectories. Meta’s SWE-RL achieved plus 10.4 points on SWE-bench Verified with a 32B parameter model through self-play, and Anthropic’s Claude Opus 4.7 used similar RL-based training for coding capabilities 2026 guide. This is the most comprehensive form and the least accessible. It requires training infrastructure few teams possess.
In mid-2026, when someone says “self-improving coding agent,” they most likely mean Layers 1 and 3 combined: loop-based iteration instrumented with telemetry. Disambiguating which layer is under discussion is the most important step before any architectural or benchmarking conversation.
What Is the Difference Between the Agent Harness and the Underlying LLM?
The agent harness (Claude Code, Codex CLI, Gemini CLI) is the orchestration layer. It manages prompts, tool calls, context windows, iteration logic, subagent spawning, and safety guardrails. It is software. Configurable, auditable, improvable by engineers.
The underlying LLM (Claude Opus, GPT-5.3, Gemini) is the reasoning engine. It generates code, analyses errors, proposes fixes. Its weights are frozen. It does not learn from one session to the next unless explicitly fine-tuned through RL.
Most performance gains in 2026 come from harness improvements rather than model upgrades. As Arize AI puts it, reliability improvement is “less about improving the model and more about improving the harness.” Better context management, smarter retry logic, and more precise tool definitions yield larger gains than model upgrades for most teams.
Sebastian Raschka makes the same point from a different angle: in many real-world applications, the surrounding system (tool use, context management, memory) plays as much of a role as the model itself. “This also helps explain why systems like Claude Code or Codex can feel significantly more capable than the same models used in a plain chat interface.”
Improvement claims that originate in the harness are different in kind from claims about model capability. Conflating them produces misleading benchmarks and inflated expectations — a pattern visible across the full self-improving agent landscape.
Both agent skills and MCP servers extend what the harness can do, but they take opposite approaches to security and auditability.
Skills (SKILL.md) are bundles of prompts and workflows loaded in-process with full harness access. MCP (Model Context Protocol) runs tools in separate server processes, each with its own runtime and credential scope comparison. On auditability, skills are opaque bundles while MCP has defined interfaces. On versioning, skills drift as markdown files evolve; MCP servers can be pinned to specific versions. On supply-chain risk, Cisco’s AI Defense team found that 26 percent of 31,000 agent skills analysed contained at least one vulnerability. For production coding agents, MCP offers the stronger security posture; skills offer greater flexibility.
OpenCode, at 172,198 stars, serves as the primary open-source harness for contrast with proprietary implementations.
What Is the Continuous Coding Loop and Why Does It Matter?
The continuous coding loop, popularised as the “Ralph Wiggum technique” by Geoffrey Huntley and Ryan Carson, follows a simple pattern: the agent picks a task, implements it, runs tests, commits if checks pass, resets context, and repeats. The name captures the spirit: “I’m helping!” becomes “I’m debugging!” becomes “I’m helping!” in an endless cycle.
Unlike single-shot code generation, this approach lets the agent compound fixes. Resolve a compilation error, then a test failure it exposed, then a lint warning the fix introduced. Converge on working solutions autonomously without human intervention between attempts self-improving agents.
The generalisation is the Agentic Loop (Observe, Plan, Act, Repeat), which underpins Claude Code, Codex, and Gemini CLI alike. All agent behaviour emerges from structured iteration. Compound looping is the goal: multiple iterations build on each other rather than resetting. Without it, the loop is just retry.
Context bloat is the limiting factor. Every iteration appends to the context window, and language models drop 15 to 30 percent accuracy on content positioned in the middle of their context. Even million-token context windows merely delay the problem. The Ralph Wiggum technique addresses this by periodically resetting context between tasks: each iteration spawns a new agent process with a clean window, reads specs from disk, takes one task, implements it, and exits.
Some architectures split planning and execution across multiple agents (the Planner-Worker Model) rather than running a single loop, trading coordination overhead for context isolation per role. Simple loops beat elaborate orchestration, but context isolation has its place.
What Is AGENTS.md and How Does It Give Coding Agents Persistent Memory?
AGENTS.md (Anthropic’s CLAUDE.md variant) is a project-level markdown file that persists conventions, architectural notes, coding preferences, known gotchas, and accumulated learnings across stateless agent sessions. Under Linux Foundation AAIF governance, it has been adopted across tens of thousands of projects.
Without persistence, each agent session starts with no memory of prior work. The continuous coding loop may improve within a session, but those improvements vanish when context is reset. AGENTS.md closes that gap. Corrections and discoveries written to disk during one session are re-injected as context in the next long-term memory. Each improvement becomes the baseline for future work.
Eric J. Ma describes the practical loop well: the agent makes a mistake, you intercept and provide the correct approach, and you instruct the agent to write the correction into AGENTS.md. “If I have to repeat the same preference every session, I am not using an agent. I am babysitting a very fast intern.” build self-improving
The practical rule: whenever you find yourself giving the same instruction twice, add it to AGENTS.md instead.
How to choose the right persistence mechanism? Simple AGENTS.md files suffice for team conventions, coding standards, and project-specific architecture notes. This is the default for most teams. External vector stores with RAG (Mem0, Zep, Letta, Augment’s Cosmos) become necessary for large codebases where AGENTS.md alone would exceed context limits. Dedicated memory persistence services are needed for multi-agent workflows where memory must be shared across planner, worker, and reviewer agents agent learning flywheel.
The hard part is not writing AGENTS.md. It is keeping it accurate as the codebase evolves. One repo review found 178 different lines between coexisting AGENTS.md and CLAUDE.md files, meaning developers got different agent experiences depending on which tool they used evaluate context.
How Do Coding Agents Use Telemetry and Traces to Verify and Improve Their Output?
Persistence answers what the agent remembers. Telemetry answers whether any of it actually works.
Telemetry is the empirical foundation of self-verification. Every tool call, every prompt, every output, every error is recorded as structured traces. These traces become the dataset for improvement at both the harness level and, potentially, the model level.
Arize AI’s Phoenix positions traces as “the source code of agentic behaviour”: not what the agent’s code says it should do, but what it actually does in production. LangChain founder Harrison Chase articulated the shift: “in software, the code documents the app; in AI, the traces do.” Agent behaviour is emergent from model-harness interaction and cannot be predicted from static code analysis alone.
At the harness level, telemetry lets engineers identify failure patterns: which tool calls produce errors, which retry strategies succeed, which context states precede degraded output. They tune the harness accordingly. At the model level, successful trajectories can be collected for reinforcement learning fine-tuning, though this remains infrastructure-intensive.
Arize describes “the loop that closes”: a coding agent receives a task, instruments relevant code paths, executes changes and collects runtime telemetry, queries traces to verify behaviour, runs targeted evaluations, and iterates using trace data and evaluation feedback closing the loop. Without telemetry, there is no source of truth. Without evaluations on real traces, there is no empirical basis for claiming a change is an improvement.
TestSprite‘s open-source CLI (Apache 2.0 licence, June 2026 release) represents the first dedicated agent self-verification tooling. It runs tests against agent output before changes are proposed. This is test-driven agent verification: having tests that verify agent behaviour, not just code output. Simon Willison’s insight is that maintaining high-quality tests in your codebase leads the agent to naturally mimic those patterns.
This telemetry pipeline feeds the benchmarks that the next article in this cluster examines — the benchmark scepticism we explore in depth. Measurement is only as good as the metrics, and whether today’s coding agent benchmarks actually measure what they claim is a question worth asking.
Disambiguation is the prerequisite to evaluating any self-improving coding agent claim, whether in a product page, a benchmark leaderboard, or a conference talk. Think of the continuous coding loop as the engine, AGENTS.md as the transmission carrying your progress forward, and telemetry as the dashboard. Each layer is necessary. None is sufficient alone.
It is an architecture of four distinct layers. The ones your team can deploy today (loops, persistence, and telemetry) are software engineering problems rather than model training problems. For teams building with coding agents in mid-2026, the practical implication is straightforward: invest in the harness first. Tool definitions, context management, retry logic, AGENTS.md discipline, and telemetry instrumentation. Before you pin hopes on the next model release, get the software infrastructure right. For an overview of what self-improving agents mean for engineering practice, including the full cluster on benchmarks, code review, and security, see the series overview.
Frequently Asked Questions
Do self-improving coding agents actually learn from my codebase?
Not in the way people assume. The underlying model weights do not update from your code unless you are running expensive RL fine-tuning (Layer 4), which almost no team does. What does improve is the agent harness: better tool selection, smarter retry logic, accumulated AGENTS.md conventions, and telemetry-driven prompt tuning. The agent gets better at working with you, not from absorbing your code into its brain.
Which coding agent should I choose: Claude Code, Codex CLI, or Gemini CLI?
The right choice depends on your stack, not the benchmark leaderboard. Claude Code offers the most mature AGENTS.md persistence and telemetry integration. Codex CLI (open-source, Apache 2.0) gives you auditability and self-hosting control. Gemini CLI shines on Google Cloud workflows with tight Vertex AI integration. If you value harness transparency over polished UX, start with Codex. If you want the most refined self-improvement loop out of the box, choose Claude Code.
Can a self-improving coding agent accidentally break my production code?
Yes, and this risk is higher than most documentation admits. All four layers of self-improvement operate on your codebase with tool access. The continuous coding loop can compound a bad fix into a worse one before you catch it. AGENTS.md conventions can drift into harmful patterns if not reviewed. The practical safeguard is test-driven agent verification (TestSprite’s CLI approach): have tests that validate agent behaviour, not just code output, before any change reaches production.
What happens when the context window fills up mid-task?
The agent hits what practitioners call the “context wall.” Reasoning quality degrades sharply (15 to 30% accuracy drops on content buried in long contexts, the “lost in the middle” problem), and the agent may start repeating earlier attempts or hallucinating fixes. This is why the Ralph Wiggum technique and other multi-session patterns periodically reset context between tasks. Without context management strategies (staged compaction, observation masking, prompt caching), the loop runs until it exhausts available tokens, then fails silently or produces degraded output.
Is AGENTS.md a security risk? What if someone tampers with it?
Yes, AGENTS.md is an in-process injection vector. Because it is loaded as context with full harness access, a compromised AGENTS.md can redirect tool calls, leak environment variables, or poison conventions across all sessions. The ClawHub marketplace identified 341 malicious skills packages exploiting this same trust boundary in 2026. Mitigation: version-control your AGENTS.md, review diffs before merging, and treat it with the same security posture as any executable configuration file in your repository.
Do I need machine learning expertise to set up a self-improving agent?
For Layers 1 through 3 (the layers that cover 95% of current deployment), you need software engineering skills, not ML expertise. Configuring the harness, writing AGENTS.md conventions, and interpreting telemetry traces are all standard engineering tasks. The loop itself (generate, execute, read errors, fix) requires no model knowledge at all. Layer 4 (model-level RL fine-tuning) is the only mechanism that demands ML infrastructure, and it remains rare outside research labs.
How can I tell if my agent is actually improving over time?
Harness-level telemetry gives you the answer, not intuition. Track three signals: error-to-resolution loop count per task (is it trending down?), first-attempt success rate on standardised tasks (is it trending up?), and manual intervention frequency (are you typing fewer corrections per session?). Arize Phoenix and similar observability platforms make these traces queryable. Without telemetry, “improvement” is anecdotal. With it, you can calculate whether your harness tuning is producing measurable gains.
Can self-improving coding agents work with languages other than Python and JavaScript?
Yes, but unevenly. The continuous coding loop is language-agnostic (it reads stdout, exit codes, and test output regardless of language). What varies is tool ecosystem quality: Python’s mature linters and test frameworks give the agent richer feedback signals than a language with weaker tooling. AGENTS.md conventions can encode language-specific patterns for any stack. The limiting factor is the underlying model’s training distribution, which still skews toward Python, TypeScript, and Go.
How is a self-improving coding agent different from an IDE copilot?
An IDE copilot (Copilot, Cursor) augments your typing with suggestions inside a single editing session. A self-improving agent operates autonomously across the full development lifecycle: it generates code, executes it, reads test results, debugs failures, writes AGENTS.md conventions, and spawns subagents for parallel work. The copilot assists a human who is driving. The agent drives itself through a continuous loop, with the human moving to a review and steering role.
Should a human still review code produced by a self-improving agent?
Absolutely, and the review model shifts rather than disappears. Instead of line-by-line code review, you review the agent’s output against your AGENTS.md conventions, verify that test coverage is adequate, and check that the agent did not introduce architectural drift across multiple sessions. The agent automates the generation and debugging loops. Human judgment remains essential for design decisions, security-sensitive paths, and the conventions that compound across sessions. Skip the review and you are skip-testing a production system.
What is compound looping and why is it harder than it sounds?
Compound looping means each iteration builds on all prior learnings rather than resetting. It sounds straightforward: just keep the context. The reality is that context bloat degrades reasoning quality, and without AGENTS.md persistence, improvements vanish between sessions. Achieving true compound looping requires solving three problems simultaneously: context management (what to keep, what to discard), persistence (AGENTS.md as the written memory), and multi-session continuity (the Ralph Wiggum technique’s reset-restart pattern). Most deployed systems manage one or two of these. Few manage all three.
