AI Coding Tools and the Code Review Bottleneck

Fiona Chen and James Stratton, two economists at Harvard, have written a paper that I think anybody leading an engineering team should take the time to read. It uses work data from 718 companies to ask what happened to software output and jobs once AI coding tools arrived, and what they found is that people wrote a lot more code without a matching increase in software shipped.

Bottlenecks in Software Production

The paper is called Artificial Intelligence in the Firm: Bottlenecks in Software Production, and it’s a job market paper, so it’s still a draft rather than a finished publication. The data comes from Jellyfish, an engineering analytics platform, and covers around 726,000 workers and roughly 300 million work events from GitHub, Jira and Google Calendar between January 2021 and March 2026.

Fiona and James compare companies that adopted AI coding tools earlier with companies that adopted them later. The timing is mostly down to how long procurement, legal and security reviews take, which they argue is unrelated to how fast a company is growing. They look at two generations of tool separately: assistants such as GitHub Copilot and Cursor, and agents such as Claude Code, which take a multi-step task and work through it largely on their own.

Writing Code Got Faster

Agents made a clear difference. Per engineer, lines of code added went up 30%, commits went up 20% and pull requests went up 23%. Assistants showed smaller increases of 12%, 9% and 5%, and only the commits figure was statistically significant. If you only looked at those numbers, you’d say the tools had worked.

Shipping Didn’t

The paper then looks at what reached production, using Jira issues and epics marked as resolved. For both kinds of tool there’s no statistically significant change. For agents, the statistical range rules out an output gain of more than 12%, which is well short of the 30% increase in code written. The authors also checked whether teams had simply moved to bigger tickets, using a model to predict ticket size, and found no sign of it.

Employment looks the same way. Overall headcount shows effectively no change after adoption, ruling out a fall of more than 2.9% for agents, and the engineering-only figures are noisier but also don’t show a drop. There’s no difference between junior and senior staff either, which differs from some other research that has found falling employment for junior workers.

Review Is Where It Piled Up

The explanation the authors give is that writing code is only one stage. Their estimate is that engineers spend somewhere between 30% and 42% of their time coding, depending on whether you use the Jellyfish data or a survey of 100 engineers, so the rest of the work stays put when the coding step speeds up.

Code review is where it shows. After agent adoption, the average time to get a pull request merged went up 49%, from about seven days to a little over ten. The share of pull requests where a reviewer asked for changes nearly doubled, from 13% to around 25%, and comments per pull request went up 35%. Pull requests didn’t get bigger, so that isn’t simply larger changes taking longer. More people also get pulled into reviewing: the share of workers doing at least one review went up 14%, while the share writing code stayed flat.

One of the interviews in the appendix really chimed with what I’ve seen myself. A junior engineer described a project where only one senior engineer could approve changes and he could only get to them once a week. After some outages caused by AI-generated commits, new rules required senior sign-off for AI-generated code, which made the queue longer still.

AI review tools haven’t fixed this yet. Around 80% of firms had used an AI code review tool by March 2026, but AI wrote only 23.3% of review comments, and only 10.8% of pull requests received an AI comment at all. Humans are still doing most of the reviewing.

We’ve seen something similar to this at Bloom & Wild. Our review backlog increased dramatically as the team started to adopt AI tools and agentic ways of working. We saw the number of people raising PRs also increase as engineering adjacent disciplines like Product Managers started to commit code to our production codebases. Our solution was to write our own code review tool, Flora, which codified our expectations and crucially also got buy-in from our engineers since it was homegrown. Reviews first go to Flora then to a human if the compexity is high or the area of the codebase where changes have been made is deemed risky. By adopting a way of working that reduces the overall number of PRs that reach a human for review, as well as ensuring that human reviews are there to catch the bugs and architectural issues that AI can’t (yet?) find effectively.

The Model

The authors build a small model of software production to make sense of this. AI changes two things: how much code gets written, and how likely that code is to contain bugs. Eliyahu Goldratt’s Theory of Constraints, from his book The Goal, would predict the first part: if you speed up a step that isn’t the constraint, the constraint stays where it is. A lot more code arriving at a fixed number of reviewers means a smaller share of it makes it through.

The second part is what I find more useful. The authors read the rise in changes requested as evidence that the bug rate has gone up as well. If AI only makes coders faster, a firm can move people into review and recover much of the gain. If AI also raises the bug rate, each piece of code needs more scrutiny, and moving people around only goes so far because the technology itself has to get better. The model also says that staffing shifts toward review relative to coding, which matches what they see in the data.

Care Needed?

The paper isn’t a randomised trial. Larger firms adopted earlier, so the authors control for company size, and they say any remaining bias would probably overstate the effects. The employment result is short-run and at the level of a single firm, so if a company gains from AI by taking business from competitors, a flat number for that company could hide job losses elsewhere. Output is measured through Jira, which depends on how consistently teams close their tickets. The agent tools are also very new in this data, with up to twelve months of results shown after adoption, and the tools are changing quickly. I’d also guess that companies that pay for an engineering analytics platform already care about measuring how their teams work, which may not describe everyone.

Things That Are Worth Thinking About

A few things that are worth thinking about:

• Lines of code, commits and pull requests all measure the stage that sped up. The Jira resolution numbers measure whether anything reached production, and in this paper those two sets of numbers tell very different stories.
• If agents mostly add volume, then moving people into review helps. If they also add bugs, the review work per pull request goes up as well, and that’s a harder problem to staff around.
• Review capacity often sits with a handful of senior engineers. Who’s allowed to approve a change is now a throughput question as well as a quality one.
• Around 80% of firms use an AI review tool, yet AI wrote under a quarter of review comments. I’d like to know what’s holding that number down.