Skip to content
Epic Software Labs
All trends

Product Engineering

Agentic coding: what changes when AI writes most of the code

What agentic coding changes for engineering teams — the METR and DORA evidence, where agents fail, and how to fix review before it becomes the bottleneck.

9 min readEpic Software Labs

Key takeaways

  • Agentic coding is the shift from autocomplete to delegation: you describe an outcome, an agent plans, edits multiple files, runs tests and iterates until the task passes.
  • Measured and perceived productivity diverge sharply. In METR's randomised trial, experienced developers were 19% slower with AI while believing they were 20% faster — so self-reported gains are not evidence.
  • DORA's 2025 data shows AI adoption correlating with higher throughput and higher delivery instability at the same time. The gain is real; so is the cost.
  • The bottleneck moves from writing code to specifying, reviewing and verifying it. Teams that do not strengthen review and testing get faster at producing work they cannot trust.
  • Agents are strongest on well-specified, well-tested, conventional code and weakest on ambiguous requirements and novel architecture — the inverse of where senior engineers add value.
  • The highest-leverage investment is not a better model. It is a codebase an agent can navigate: fast tests, clear boundaries, documented conventions and reproducible environments.
  • Measure throughput and stability together. Speed gains that raise change failure rate are not gains.

Agentic coding is the point at which AI stops suggesting lines and starts completing tasks — and most engineering teams have now used it without changing how they work because of it. That gap — between using the tools and restructuring around them — is where the actual productivity difference sits, and it is why two teams with identical tooling can end up on completely different trajectories.

This guide is about the second thing.

What agentic coding actually means

The first wave of AI coding tools was completion: you type, the model predicts, you accept or reject. The interaction was tight, the unit of work was a line or a block, and you stayed in control of every keystroke because you were still doing the typing.

Agentic coding inverts that. You hand over a task — "add rate limiting to the public API and cover it with tests" — and the tool does the loop itself: reads the relevant files, forms a plan, makes edits across the repository, runs the test suite, reads the failures, and tries again. You review the result rather than the process.

The difference matters more than it sounds. Completion made individual engineers slightly faster at a task they were already doing. Delegation changes what the job is.

What the evidence actually shows

The most useful thing about this trend is that it is now old enough to have a research literature, and the literature is more interesting than the marketing.

Perceived speedup and measured speedup are not the same quantity. The METR study of AI productivity is the most careful measurement available, and its headline result is uncomfortable. METR ran a randomised controlled trial with sixteen experienced open-source developers across 246 real tasks drawn from their own backlogs, randomising whether AI tools were permitted per task. Developers completed tasks 19% slower when allowed to use AI — while estimating afterwards that they had been 20% faster (arXiv:2507.09089). The population matters: these were maintainers working in repositories they knew deeply, which is close to the worst case for a tool whose main advantage is orientation speed in unfamiliar code. METR have since revised their experiment design (February 2026), and the result should be read as one careful measurement of one population, not a universal law.

At organisational scale, throughput rises and so does instability. DORA's 2025 research found roughly 90% adoption among technology professionals, with AI adoption correlating positively with software delivery throughput and positively with delivery instability — more change failures, more rework (DORA). Their reading is that AI exposes downstream bottlenecks in testing, review and QA rather than creating new problems of its own.

Put those two findings together and the picture is coherent: generation accelerated, verification did not, and self-report is an unreliable instrument for measuring the difference.

Where the bottleneck moves

Every process improvement relocates the constraint rather than removing it. Agentic coding relocates it from production to verification.

When code took a week to write, review was cheap by comparison and a slow review process was invisible. When the same change arrives in an hour, the review queue becomes the dominant cost — and the review itself is harder, because the reviewer has none of the context that comes from having written the thing.

Three failure patterns follow from this, and they are worth naming because they are so common:

Review theatre. Diffs get bigger and arrive faster, so reviewers skim. Approval rates stay high, defect rates climb, and nobody notices for a quarter because the dashboard only measures throughput.

Confident wrongness. Agents produce code that looks idiomatic and reads well. Plausibility is not correctness, and well-formatted code with clean naming triggers less scrutiny than messy code doing the same wrong thing.

Architectural drift. Each individual change is reasonable in isolation. Nobody is holding the shape of the system in their head, because the person who would have been holding it was reviewing diffs instead of writing them.

None of these are arguments against agents. They are arguments for changing what you invest in when you adopt them.

What agents are good at, and what they are not

The pattern is consistent enough to plan around.

Task typeAgent performanceWhy
Well-specified changes in a conventional codebaseStrongPattern is visible in surrounding code; success is checkable
Test writing and coverage backfillStrongThe target is explicit and verification is mechanical
Refactors with a clear invariant ("extract this, keep behaviour")StrongTests provide the signal to iterate against
Migrations across many filesStrongRepetitive, mechanical, tedious for humans, easily verified
Debugging with a reproducible failureGoodFast feedback loop; the agent can test hypotheses
Integrating an unfamiliar third-party APIMixedDepends heavily on how well documented it is
Ambiguous requirementsWeakFills gaps with plausible assumptions rather than asking
Novel architecture decisionsWeakOptimises for the common pattern, not your constraints
Performance work without a profilerWeakGuesses at hot paths; the feedback signal is missing
Security-sensitive boundariesWeak, and expensive to get wrongSubtle, adversarial, poorly represented by tests

Read the two halves of that table together and the strategy becomes obvious. Agents are strongest exactly where senior engineers add the least value, and weakest exactly where senior engineers add the most. That is a good trade — but only if you actually reallocate senior attention rather than pointing it at the same work faster.

The real prerequisite is your codebase

There is a persistent belief that the next model release will fix whatever is not working. Sometimes it does. But the largest variable in how well agentic coding works is not the model — it is whether your repository gives an agent a way to tell whether it succeeded.

An agent working without a feedback signal is guessing. An agent with a fast, reliable test suite is iterating. Those produce completely different quality outcomes from the identical model.

In rough order of impact:

  1. Fast, trustworthy tests. This is the single biggest lever. A suite that runs in ninety seconds and fails only for real reasons turns an agent into a system that converges on correct. A twenty-minute suite with three known-flaky tests turns it into a system that produces confident guesses.
  2. A reproducible environment. If setup is a wiki page of manual steps, the agent cannot run anything, and you are back to review-by-inspection.
  3. Clear module boundaries. Agents follow the structure they find. Well-separated modules keep the blast radius of any single change small and reviewable.
  4. Documented conventions. A file that states how this codebase does error handling, logging and data access is read and followed. Conventions that live only in reviewers' heads get violated on every change.
  5. Meaningful names and types. Both a strong type system and honest naming narrow the space of wrong answers before the agent generates anything.

Restructuring the workflow

Teams that get real leverage tend to converge on similar practices, arrived at independently.

Specification gets written down. Not a formal document — a paragraph that states the goal, the constraints and how you will know it worked. This was always good practice. With agents it becomes load-bearing, because a vague brief now produces a confidently wrong implementation in ten minutes instead of a clarifying question on day two.

Changes get smaller, not larger. The temptation is to hand over bigger tasks because the agent can handle them. Resist it. Review capacity is the constraint, and review quality falls off a cliff somewhere around a few hundred lines regardless of who wrote them.

Tests come first, and from a different source. The strongest pattern we have seen: a human writes or approves the failing test that defines "done", then the agent makes it pass. This closes the loop where an agent writes both the implementation and the test that blesses it.

Someone owns the architecture explicitly. When nobody is writing the code end to end, coherence stops being a free by-product of authorship. It has to become a named responsibility.

What a well-run agentic workflow looks like day to day

Descriptions of this trend tend to stay abstract. Here is the concrete shape of a working loop, as run by teams who have made it stick.

The task arrives specified, not described. Someone has written a paragraph stating the goal, the constraints, and how anyone will know it worked. Where that paragraph does not exist, writing it is the first task — and it is human work. This is the point where spec-driven development and agentic coding meet: the discipline that felt like bureaucracy when code was expensive becomes the thing that makes delegation possible.

A failing test exists before the implementation. Written or at minimum approved by a person. This is the single highest-value practice in the whole workflow, and the one most often skipped. Without it, the agent writes both the code and the test that blesses it, and you have a system marking its own homework in a convincing hand.

The agent runs against a checkout, not a shared environment. Credentials scoped to the minimum. No write access to production or to external services without a human in the loop. The risk model is closer to onboarding a fast, confident contractor on their first day than to running untrusted code, and the controls should look like it.

The diff comes back small. Teams that get value here cap change size deliberately, usually somewhere between two and four hundred lines. The temptation runs the other way — generation is cheap, so why not hand over more? Because review capacity is the constraint, and review quality falls off a cliff past a few hundred lines regardless of who did the typing.

Review reads the boring parts. Human authors make mistakes where the problem was hard. Agents make mistakes where the requirement was ambiguous, which means the section of a diff that looks most routine is often where an assumption was quietly baked in. This inverts twenty years of reviewer instinct and takes deliberate practice to learn.

Someone owns the shape of the system. When nobody writes the code end to end, architectural coherence stops being a free by-product of authorship. It has to become a named responsibility held by a person, reviewed continuously, or the system becomes fifty locally-reasonable decisions with no through-line.

The team behaviours that predict failure

Three patterns show up reliably before the numbers do:

  • Approval latency falling while diff size rises. The clearest early signal that review has become a formality. Most teams do not instrument it, which is why they find out from their change failure rate a quarter later.
  • Nobody able to explain a critical path. Ask who could debug the payment flow at 3am without the original author. If the honest answer is nobody, generation has outrun comprehension.
  • The test suite getting slower and no one minding. A slow suite removes the agent's feedback signal and turns iteration back into guessing. Teams that let this slide lose most of the benefit without noticing where it went.

The counterpart to this from a governance angle — the questions to ask if you are funding the work rather than doing it — is covered in is AI-generated code safe to ship.

Measuring it honestly

The most common mistake in evaluating this shift is measuring only the half that improved.

DORA's research is useful here precisely because it insists on measuring throughput and stability together: deployment frequency and lead time on one side, change failure rate and time to restore on the other. Their 2025 finding — adoption correlating with both higher throughput and higher instability — is what happens when an organisation measures only the first pair and optimises accordingly.

Note also that the correlation runs in an unhelpful direction for causal claims: teams that adopt AI aggressively differ from teams that do not in many ways besides AI. Treat these as signals to instrument your own delivery, not as effect sizes you can expect to inherit.

Track these together, and treat a rising change failure rate as the signal it is:

  • Lead time for changes — should fall
  • Deployment frequency — should rise
  • Change failure rate — should hold flat or fall; if it rises, your review and test investment has not kept pace
  • Time to restore service — should hold flat
  • Review latency — the new bottleneck, and the one most teams do not instrument at all

If throughput is up and change failure rate is up, you have not become faster. You have moved work from before the release to after it.

Adopting this without breaking what works

We embed with engineering teams to restructure around agentic workflows — test infrastructure first, review practice second, tooling last. The first conversation is usually about your test suite.

Start a project

What to do in the next quarter

If you want a concrete sequence rather than a set of principles:

  1. Measure your test suite. Wall-clock runtime and flake rate. If it is slow or unreliable, that is the project — everything else is downstream of it.
  2. Pick one well-bounded surface and run agentic workflows there for a month. A service with good tests and clear boundaries. Not the payments core, and not the legacy monolith.
  3. Write the conventions file. Error handling, logging, data access, testing style. You will find that new human hires benefit from it just as much.
  4. Tighten review before you scale up volume. Smaller diffs, mandatory failing-test-first, explicit ownership of the architecture.
  5. Instrument review latency. It is about to become your constraint and almost nobody is watching it.
  6. Review the DORA four together, monthly. Speed alone is a vanity metric.

The teams pulling ahead are not the ones with early access to the best model. They are the ones who spent the last year making their codebase legible — to agents and to people, which turn out to be nearly the same requirement.

Frequently asked questions

What is the difference between agentic coding and AI autocomplete?

Autocomplete suggests the next few lines inside the file you are editing, and you accept or reject each suggestion. An agentic tool takes a goal instead of a cursor position: it reads the repository, plans a change, edits several files, runs the test suite or build, reads the failures and tries again. The unit of work is a task, not a line.

Do coding agents replace software engineers?

They replace a portion of the typing, not the engineering. Agents are effective when the requirement is unambiguous, the conventions are visible and there is a fast feedback signal such as a test suite. Deciding what to build, choosing an architecture that will survive the next two years, and judging whether generated code is genuinely correct all remain human work — and they become a larger share of the job, not a smaller one.

How do you review code an agent wrote?

Review the diff on its merits and ignore its provenance. In practice this means insisting on smaller changes, requiring the agent to produce tests that fail before the fix and pass after, and paying disproportionate attention to the areas agents are weakest on: error handling, concurrency, security boundaries and anything involving money or personal data.

Is it safe to let an agent run commands in a repository?

It is safe when the blast radius is bounded. Run agents against a checkout rather than a shared environment, scope credentials to the minimum needed, require human approval for anything that writes to production or external services, and keep every change behind normal code review and CI. The risk model is closer to onboarding a fast, over-confident contractor than to running untrusted code.

Does research show AI coding tools actually make developers faster?

The evidence is mixed and depends heavily on who is measured. METR's randomised controlled trial found experienced open-source maintainers were 19% slower on tasks in repositories they knew well, while self-reporting a 20% speedup. DORA's 2025 industry data, by contrast, found AI adoption correlating with higher delivery throughput across organisations — alongside higher instability. The reconciliation is that AI helps most where orientation is expensive, which is precisely where deep familiarity already removed that cost, and that speed gained in generation can be lost again in review and rework.

What kind of codebase works best with coding agents?

The same one that works best for a new human hire: fast and reliable tests, clear module boundaries, consistent conventions, a reproducible local setup and documentation that says why things are the way they are. Agents amplify whatever your codebase already is. A well-structured repository gets faster; a tangled one produces plausible changes that quietly break things.

References & further reading

  1. [1]
  2. [2]
  3. [3]
  4. [4]
  5. [5]
  6. [6]
  7. [7]