Agentic Coding: How AI Coding Agents Work and What They Cost

Updated Nov 21, 2026 · 11 min read · ntimer

Agentic coding means delegating software work to an AI coding agent that reads your codebase, edits files, runs commands and checks its own output in a loop while you steer and review it. The measured results are mixed: one controlled trial found a 55.8% speed-up on a scoped task, another found experienced developers 19% slower, and a 2026 update lands in between. This guide covers the loop, the tools, the research, and how to measure an agent's time.

What is agentic coding?

Agentic coding is building software by directing an AI coding agent that acts on your repository: it reads files, edits code, runs commands and verifies results, while you set the goal and review the output. Claude Code is described by its own documentation as an agentic coding tool that reads your codebase, edits files and runs commands [1].

The practical difference from autocomplete is agency. An inline assistant proposes the next line and stops; an agent keeps acting until the task looks done. Anthropic's documentation describes the loop as three blending phases: gather context, take action, verify results [2].

Tools are what make it agentic: they turn a model into something that can act on the world [2]. It also chains many steps without a new prompt for each one. That is also why time measurement changes: one task can hold a terminal open for an hour while your own typing takes ten minutes. Agentic coding keeps the checks; vibe coding drops them.

How does an AI coding agent loop work?

Most agents run the same five steps in a loop: gather context, plan, edit, run, check. Anthropic's Claude Code documentation describes the loop as three blending phases, gather context, take action and verify results, and shows the pattern in practice: run the tests, read the error, search for the source, edit, then run the tests again [2].

  1. Gather context. Search the codebase, read the relevant files and run the test suite to see what is actually failing [2].
  2. Plan. Turn the goal into concrete steps. Claude Code breaks complex work into steps and adjusts as it learns, and Cursor ships a Plan Mode for scoping changes first [2][6].
  3. Edit. Change files, often several at once. The built-in tools cover reading, editing, creating and reorganizing files [2].
  4. Run. Execute commands, tests and builds on your machine instead of only suggesting code [1][4].
  5. Check. Read the output and verify. Without a command that proves the work is done, the agent stops when the work looks done [3].

The loop repeats until the check passes or the agent gives up, and you can interrupt it at any point to steer it or ask for a different approach [2]. If you want the agent to drive itself in the background, the Claude Code MCP guide covers how the same loop is wired to external tools.

Agentic coding vs autocomplete vs chat vs vibe coding

Autocomplete suggests the next line, chat answers a question, agentic coding changes files and runs commands until the task is done, and vibe coding is agentic coding with the review step removed. The styles blur in practice, but the review step is the difference that matters for anything you ship.

How the four styles differ
AutocompleteChatAgentic codingVibe coding
Who actsSuggests a lineExplainsActs on the repoActs, you accept the output
Uses toolsNoUsually noYes: files, shell, tests [2]Yes
Verifies its own workNoNoYes, if you give it a check [3]No
You read the diffOftenNot applicableYes, before merging [3]Rarely [8]
Main riskSlow reviewConfident errorsUnreviewed changes at scaleCode nobody understands [8][9]

That risk is not hypothetical. In the Stack Overflow 2025 survey, more developers distrust the accuracy of AI tools (46%) than trust it (33%), and only 3% trust the output highly [14].

What is vibe coding, and where does the term come from?

Vibe coding is shipping AI-written code without reading it, on the idea that the model is good enough to iterate until it works. The term was coined by Andrej Karpathy on 2 February 2025 [8][9]. Merriam-Webster later listed it as a slang and trending expression, and Collins named it its Word of the Year for 2025 [9].

I just see stuff, say stuff, run stuff, and copy paste stuff

[9]

Karpathy described the method honestly: accept every change, paste error messages back without comment, and let the code grow past what you can hold in your head. He said it is not too bad for throwaway weekend projects [8].

That is the whole difference from agentic coding. Vibe coding removes the check; agentic coding adds one: you hand the agent a test to run, ask for the command and its output as evidence, and read the diff before it merges [3]. Vibe coding is fine for exploring an idea and poor for shipping code you must maintain, because you are the one who will debug it later. Wikipedia's summary of the debate carries the same warning, quoting Simon Willison that vibe coding your way into a production codebase is clearly risky [9].

Which AI coding agents are there?

The four you will meet most often are Claude Code, OpenAI Codex, Cursor's agent and GitHub Copilot's coding agent. All four can act on a repository. They differ in where the work runs, your machine, an editor or a cloud sandbox, and in how much of the loop they own.

What each tool says it does, in its own words
ToolWhat it isWhere the work runs
Claude CodeAn agentic coding tool that reads your codebase, edits files and runs commands; its tools cover files, search, execution and web [1][2]Terminal, IDE, desktop app and browser [1]
OpenAI CodexCodex CLI inspects files, makes edits and runs the tools installed on your machine, and runs from scripts with codex exec [4]Your terminal, or Codex Cloud, where each task has its own workspace and keeps working while your computer sleeps [5]
Cursor's agentA coding agent that understands your codebase, plans features, fixes bugs and reviews changes [6]Your editor, with Plan Mode for scoping a change [6]
GitHub Copilot's coding agentCopilot researches a repository, plans changes and implements them in the background, then you review them and create a pull request [7]A cloud development environment, not your local IDE [7]

Two differences matter for your review process and your invoice: where the work runs, and whether it keeps running on its own. A cloud agent keeps going after you close the laptop, which is convenient and also means the clock moves when nobody is watching [5].

Does agentic coding actually make developers faster?

It depends on the task and on how you measure. A 2023 controlled experiment found developers finished a scoped JavaScript task 55.8% faster with Copilot [10]. A 2025 randomised trial found the opposite for experienced open-source developers, who took 19% longer [11]. A 2026 update lands in between [12].

  • 55.8% faster, one task. Peng and colleagues randomised developers to implement an HTTP server in JavaScript with or without GitHub Copilot; the treatment group finished 55.8% faster [10]. The scope matters: one greenfield task, early Copilot, recruited developers.
  • 19% slower, real repositories. METR randomly allowed or disallowed AI on 246 real issues from the repositories of 16 experienced open-source developers. With AI allowed, tasks took 19% longer. Developers expected a 24% speed-up, and still believed afterwards they had been sped up by 20% [11].
  • In between, and messy, in 2026. METR's second study was undermined by selection effects: more developers refused to work without AI, and 30% to 50% of developers said they skipped some tasks they would not do without AI. For the returning developers, task time fell by an estimated 18%, but the confidence interval (-38% to +9%) includes no effect, and METR calls it weak evidence [12].
  • 1.5x to 13x, and a ceiling rather than a floor. A February 2026 note analysed 5,305 Claude Code transcripts from seven METR staff and estimated a time-savings factor of roughly 1.5x to 13x. The author calls it a soft upper bound: people pick the tasks they expect AI to win, and agents add overhead of their own [13].
  • What developers report. In the Stack Overflow 2025 survey, 84% of respondents were using or planning to use AI tools, 51% of professional developers used them daily, and 52% said AI tools or agents had changed how they work [14]. Self-report and measurement do not agree, which is the honest summary of this field.

Read those numbers as a range, not a promise. DORA's 2025 research frames AI as an amplifier: it magnifies the strengths and weaknesses of the organisation around it [15]. A team with fast tests gets a bigger win from an agent than a team without them, and that is also where the slowdowns come from.

What are the risks, and which review practices actually work?

The risk is not that agents write code, it is that plausible-looking code arrives faster than anyone can check it. The Stack Overflow 2025 survey found 66% of developers name almost-right-but-not-quite solutions as their biggest frustration [14]. The fix is a check the agent has to run.

Anthropic's best-practices guide is blunt: without a command that tells the agent whether the work is really done, looks done is all it has to go on, which in practice means the test suite [3].

  • Give it a test. Name the exact test, typecheck or lint command in the prompt or in the project's instruction file [3].
  • Ask for evidence. Ask for the command it ran and the output, not a summary: all tests pass on its own tells you very little [3].
  • Use a fresh reviewer. Run code review in a separate subagent that never saw the conversation, so it is not swayed by the reasoning behind the code [3].
  • Enforce, do not suggest. Hooks, deny rules and the sandbox apply every time, whatever the model decided to do [3].
  • Cap the spend. Effort level, --max-turns and --max-budget-usd bound how much a run can cost, and /usage shows where it went [3].
  • Review against a spec. Check the work against your plan and tell the reviewer to skip style: a reviewer asked to find problems will always find something [3].

Two habits are worth keeping even without the tooling: write the failing test before the fix, and treat any change you cannot explain as unfinished [3].

How do you measure how long an AI agent actually worked?

Measure agent time as its own category, not as your own. An agent session can run for an hour while you spend ten minutes steering it, and METR's transcript analysis found that higher agent concurrency went with higher time-savings factors, so one wall-clock number hides the parallel work [13].

  • Give the agent its own timer. Run one for the agent session and another for your review, and log the repo, the task and what was verified.
  • Keep the session alive with heartbeats. A heartbeat says the agent is working, silence says it is not. Without that signal a stalled session and a busy one look the same on a timer.
  • Cut off idle time. Close the timer when the agent goes quiet, so a session that stalls overnight adds no hours to the invoice.
  • Pair time with cost, and report a ratio. Minutes alone hide the expensive run, so log the model, tokens and turns next to the duration and set a spend limit [3]. Time to first useful change, time to a passing check and your review time are three different numbers, and clients understand all three.
Agent timers over MCP and heartbeats

ntimer's remote MCP server runs agent timers. An agent starts a timer with ntime_start for the client work it is doing, keeps it alive with ntime_heartbeat while it works, and stops it with ntime_stop. The timer closes at the last heartbeat after idleMinutes of silence, 30 by default, so waiting time is never recorded. Connect Claude Code with claude mcp add --transport http -s user ntime https://api.ntimer.app/mcp, then claude mcp login ntime; any MCP client can use the same endpoint.

Should you bill a client for time an AI agent spent?

Only if the contract allows it, and only if the client can see what happened. Bill for the outcome and the review, and show agent time as a separate line rather than folding it quietly into your hourly rate.

Three rules keep this clean. First, ask before you bill it: most contracts price a person's time, so an agent line changes the terms, not just the invoice. Second, give the client separate lines for agent run time, your review time and the total. Third, keep the agent's sessions under their own work item so they never inflate a client's total by accident.

If the agent's time is not billable, record it anyway. Measured non-billable time is the only way to see whether the agent pays for itself, and the billable hours guide covers the mechanics: clients, rate profiles, rounding rules and what an invoice must show.

Rate profiles, worklog and invoices

In ntimer, a client or project is a work in a tree with a worklog of the sessions under it. Rate profiles set the hourly rate for each work, and Free plans include three, so agent time and your own time can carry different rates on one project. On Pro you can turn a month into an invoice in EUR or USD and send it by mail.

Questions

What is the difference between agentic coding and vibe coding?

Agentic coding keeps the verification step: the agent acts on your repository and you make it prove the work with tests, typechecks or a lint command before you merge. Vibe coding removes that step, accepting whatever the model produces. Karpathy coined the term for throwaway weekend projects [3][8].

Do AI coding agents actually make developers faster?

It depends on the task and the measurement. One controlled experiment found a 55.8% speed-up on a scoped JavaScript task [10], a randomised trial on real open-source issues found experienced developers 19% slower [11], and a 2026 follow-up study put the effect between the two [12]. Treat any single number as a snapshot, not a promise.

How do I stop an agent session from running up the clock?

Keep the session alive with heartbeats rather than a plain elapsed-time counter, and close the timer when the heartbeats stop. In ntimer the agent sends ntime_heartbeat while it works, and the timer closes at the last heartbeat after idleMinutes of silence, 30 by default, so idle waiting is not billed.

Should I charge my client for time an AI agent spent?

Only with the client's agreement, and only when you can show what the agent did. Keep agent run time, your review time and the total on separate lines, and put the agent's sessions under their own work item. If the contract prices a person's time, an agent line needs a conversation first.

What can an AI coding agent do that autocomplete cannot?

Autocomplete suggests the next line of code and stops. A coding agent can search the codebase, read files, edit across several of them, run tests and builds, read the output and iterate until a check passes [1][2]. That is a suggestion versus a loop you can delegate a task to.

Sources

  1. Overview, Anthropic (Claude Code docs). accessed Nov 21, 2026. Back
  2. How Claude Code works, Anthropic (Claude Code docs). accessed Nov 21, 2026. Back
  3. Best practices for Claude Code, Anthropic. accessed Nov 21, 2026. Back
  4. Codex CLI, OpenAI. accessed Nov 21, 2026. Back
  5. Codex Cloud, OpenAI. accessed Nov 21, 2026. Back
  6. Agent overview, Cursor. accessed Nov 21, 2026. Back
  7. Copilot on GitHub, GitHub Docs. accessed Nov 21, 2026. Back
  8. There's a new kind of coding I call "vibe coding", Andrej Karpathy on X, 2025. accessed Nov 21, 2026. Back
  9. Vibe coding, Wikipedia. accessed Nov 21, 2026. Back
  10. The Impact of AI on Developer Productivity: Evidence from GitHub Copilot, arXiv, 2023. accessed Nov 21, 2026. Back
  11. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, METR, 2025. accessed Nov 21, 2026. Back
  12. We are Changing our Developer Productivity Experiment Design, METR, 2026. accessed Nov 21, 2026. Back
  13. Analyzing coding agent transcripts to upper bound productivity gains from AI agents, METR, 2026. accessed Nov 21, 2026. Back
  14. Stack Overflow Developer Survey 2025: AI, Stack Overflow, 2025. accessed Nov 21, 2026. Back
  15. Research: 2025 — State of AI-assisted Software Development, DORA (Google Cloud), 2025. accessed Nov 21, 2026. Back

Related guides

00:00:00

Start the clock.

Start tracking