An AI coding assistant may help you finish more work, but a productivity headline cannot tell you how many hours you will save. Published field experiments have found both more completed work and longer task times. They studied different people, tools and outcomes.

The useful question is where your working day goes: writing code, understanding a repository, waiting for an agent, or checking its changes. Two research projects, followed by METR’s 2026 updates, show why those distinctions matter. None provides a measured percentage for the coding agents available in October 2026.

Two experiments, two different measures

Cui and colleagues studied early GitHub Copilot rollouts at three companies. Becker and colleagues at METR studied experienced open-source developers working on familiar repositories. These are separate research projects; the figures below are their reported findings.

Historical field experiments. An increase in completed work is not the same metric as a change in task time.

Study and setting What was measured Reported result Main limit
Cui et al.: company Copilot trials beginning in 2022–2023 Weekly completed pull requests 26.08% more; standard error 10.3 percentage points An adoption-adjusted estimate, not a measurement of minutes saved
METR: familiar open-source repositories, February–June 2025 Implementation time per task 19% longer; reported confidence interval 2%–39% longer A small, experienced sample using that period’s tools

More pull requests does not mean the same percentage less time

Cui and colleagues’ February 2025 paper combines three company-run trials at Microsoft, Accenture and an anonymous Fortune 100 firm, reporting a combined sample of 4,867 developers. The trials began in September 2022, July 2023 and autumn 2023, respectively. The tool supplied code completions inside the editor; this was not a test of today’s autonomous coding agents. Exact backend model versions and token budgets are not disclosed.

The companies randomized access or invitation timing. Some developers who received access did not adopt Copilot. The authors therefore used the random assignment to estimate the effect of adoption, giving more weight to periods when adoption differed more between groups, then combining the company estimates according to their precision. The table’s result is this adjusted estimate, not the raw difference between everyone offered a license and everyone in the control group. Each company’s estimate is uncertain.

A completed pull request is a useful unit of work within a team’s conventions. It does not tell you how long a task took, how large the change was, or how much value it delivered. Splitting one feature into more requests would raise the count without necessarily improving delivery. That is a reason to pair output counts with time and quality when reviewing your own workflow, rather than translating the study’s percentage into hours saved.

Familiar code can make review costs matter more

METR randomized whether AI was allowed for each of 246 real tasks completed by 16 experienced developers. They worked on large repositories they already knew, primarily using Cursor Pro with Claude 3.5 or 3.7 Sonnet during February–June 2025. Developers reported their implementation time and recorded their screens. The headline time estimate adjusts for task difficulty forecasts made before randomization.

Before working, developers expected AI to cut their time by 24%. Afterwards, they still thought it had made them about 20% faster. Those beliefs differed from the measured slowdown. METR’s analysis points to factors including repository familiarity, missing context, unreliable suggestions and the time spent prompting, waiting and reviewing.

For a familiar bug fix, a plausible-looking change may still need substantial checking against local conventions, tests and documentation. The practical question is whether the suggestion removes more work than that checking adds. The study does not establish that most developers are slowed down, or that later tools perform the same way. Its narrow setting is useful precisely because it identifies a situation to examine, rather than supplying a universal verdict.

The 2026 follow-up is not a clean reversal

METR began a continuation in August 2025 and reported it on 24 February 2026. It included 10 returning developers and 47 new participants. Returning developers’ point estimate was 18% less time, with an interval spanning 38% less to 9% more. New participants’ estimate was 4% less time, spanning 15% less to 9% more. Both intervals include no change and some slowdown.

The authors also warned that the data were an unreliable signal of the current effect. People increasingly declined to participate because they did not want to work without AI, and participants withheld some tasks for the same reason. Participation pay fell from $150 to $50 per hour, potentially changing who joined. Running several agents while doing other work also made task-time logs harder to interpret.

Those limitations matter more than choosing whichever point estimate supports your view. The follow-up suggests conditions were changing, but it does not reliably establish the size of a present-day speedup. It comes from the same research group and partly the same participants, so it is not an independent replication. Nor does it assign a productivity percentage to a particular current version of Claude Code or Codex.

The newer survey measures what people believe

METR’s May 2026 report surveyed 349 technical workers during February–April, including 87 software engineers. Respondents reported a median 1.4–2x change in the value of their work across question wordings, and a median 3x change in speed. These are self-reported estimates from a convenience sample, not timed experimental results. The authors explicitly question how closely the reported gains match reality.

Value and speed can also diverge. Quickly producing an extra dashboard does not necessarily make the underlying project more valuable. Conversely, an assistant might make worthwhile work possible that you otherwise would not attempt. The survey brings these possibilities into view; it does not causally confirm either of the earlier experiments.

Keep a small record of the work you actually accept

Start with one recurring task class, such as small bug fixes in one repository. Decide what counts as finished before recording results. A generated patch alone is a different outcome from a tested, reviewed change that the team accepts.

The following is a proposed log for your own work, not a new experiment result. It helps expose where time goes and which changes remain useful.

  • Record the task and its completion criteria, including tests and review.
  • Separate active human minutes from elapsed time and time waiting for an agent; note overlapping work.
  • Include review and rework, then record accepted output and later defects.
  • Note the tool, model or version when available, and any workflow changes.

Use that record to ask whether comparable accepted changes require fewer human minutes without a worsening defect record. A small personal log cannot establish causality, and learning or task selection can change the result. It can still give you a better basis for adjusting your workflow than borrowing a percentage measured on someone else’s work.

Sources

Claude Radar synthesized the authors' published findings and did not rerun the experiments or reanalyze participant data. Sources checked on 3 October 2026: Cui, Demirer, Jaffe, Musolff, Peng and Salz's February 2025 paper and Microsoft Research summary; Becker, Rush, Barnes and Rein's July 2025 METR paper; and METR's February and May 2026 updates. The company trials include Microsoft-employed coauthors. An earlier abandoned Accenture trial with missing usage data is outside the headline sample. The papers and their summary pages are not separate replications. These historical experiments and later self-reports do not measure October 2026 tools.