Structured Policy Analysis

Does AI Coding Assistance Actually Speed Developers Up?

When AI coding assistance speeds developers up versus slows them down, and why developers are often wrong about which is happening. AI research grounded in evidence, structured by causal mechanisms. Independent verification required.

0claims analyzed

0sources cited

0causal mechanisms

Key Findings

Research suggests AI coding assistance can speed developers up sharply on some tasks and slow them down on others. On greenfield, self-contained tasks, one controlled experiment found developers finished about 56% faster with GitHub Copilot. On familiar, high-context code, a 2025 randomized trial of experienced open-source developers found they took about 19% longer with AI, while believing they were faster. The split appears to track developer context: new or low-context code and less experienced developers tend to gain, while experts on code they already know deeply may not. A recurring theme is that perceived speedup and measured speedup can diverge, which complicates the survey evidence the industry often cites.

Impacts vary widely depending on task type, codebase familiarity, developer experience, and which AI tool and model are used. Findings from one experiment, task, or tool generation do not necessarily generalize to others, and AI capability is changing quickly.

Greenfield speedup is real and large

A controlled experiment asked developers to build an HTTP server from scratch. The group with GitHub Copilot finished about 56% faster. Effects on isolated, well-defined tasks do not necessarily extend to whole projects.

Experts on familiar code can be slower

A 2025 randomized trial gave 16 experienced developers AI tools on their own mature repositories. They took roughly 19% longer with AI, against their own expectation of a speedup. The result is specific to high-context, high-standard settings.

Developers are often wrong about which is happening

In the same trial, developers predicted a 24% speedup and still believed AI sped them up about 20% after measured results showed a slowdown. A separate survey found 88% felt more productive with Copilot. Perception and measurement can diverge.

Less experienced workers tend to gain more

Across field experiments with thousands of developers and in a large customer support study, less experienced workers showed larger gains. More experienced workers sometimes showed little measured effect.

Individual output is not organizational output

Telemetry from over 10,000 developers found high AI adoption raised tasks completed per developer but also raised review time and bugs, with no measured gain in overall delivery. Speed at one stage can shift work to another.

Quality and security signals are mixed

One randomized trial found code written with Copilot passed more tests and scored slightly higher on quality. A security study found participants with an AI assistant wrote less secure code while feeling more confident. Results depend on task and measure.

Research Findings

Sources

What this means in practice

Work related to measuring AI's effect on a team often involves manually pulling activity from many tools, reconciling what people report against what the systems actually recorded, and assembling that into a repeatable report. These processes are typically handled with systems that automate the repetitive parts.

Ingest activity and outcome data from development and project tools
Reconcile self-reported time savings against recorded throughput and rework automatically
Generate clear, repeatable reports that separate perceived gains from measured ones

See example systems