Junior Engineers Are Doing Better Work and Learning Less from It

A study from the University of New South Wales this year gave undergraduate computer science students three introductory C programming tasks. Half the group used ChatGPT-4.5, half used ordinary web search. The AI-assisted group scored 89% on the tasks against 69% for the web-search group, a 20-point gap. Two wo days later they took a retention test which showed that the ChatGPT group scored 39%, the web-search group 52%. Asked how much of their submitted code was really theirs, the ChatGPT group put it at 45%. The web-search group put it at 81%. I think that’’s the number worth thinking about, not the first one.

The UNSW Study

Chris Stokel-Walker wrote this up for LeadDev, and it’s a small study (55 students, one university, undergraduates rather than working engineers), so I wouldn’t treat the exact percentages as gospel. But the shape of the result is the useful part: give someone an AI coding tool and their output goes up while what they can explain, defend or recall about that output goes down. I think that’s less a tooling story than a story about what we’re actually optimising for when we hand a junior engineer a task and measure how fast it comes back.

Adrian Harwood, who leads research software engineering at the University of Manchester, makes the point in the same article that banning the tools is pointless. His team keeps junior engineers close to experienced colleagues instead: nobody completes a project alone, senior engineers review every pull request including the AI-generated parts, and training now explicitly covers how to critique what the model produces rather than just how to prompt it. He calls this “how to police the tool.” I think that’s exactly the right level to operate at: the problem isn’t the tool itself, but letting its output stand in for a junior engineer’s own reasoning.

Mia de Búrca’s Argument

The UNSW study is about measurement. Mia de Búrca’s piece, also on LeadDev, is about what gets lost along the way, and I think the two pieces are really making the same case from different directions. Her argument is that junior engineers used to build judgement through friction: bad assumptions, broken code, the debugging that follows, the reconsidering of the problem that debugging forces on you. If you take that friction away, you remove not just the drag, but the thing that was actually teaching them. As she puts it, “some friction was drag. Some friction was education,” and AI tools are very good at removing both without telling you which one they just took.

What I like about her piece is that she doesn’t treat this as unfixable. Her suggestions are concrete: make critical thinking an explicit expectation rather than an assumed one, since a recent graduate genuinely may not know that reasoning is part of the job rather than a bonus on top of it. Build enough psychological safety that an engineer can say “I don’t understand what the AI generated” without it reading as a confession. Ask for pull requests that explain why a change was made, not just what changed. Use postmortems to interrogate what the team believed that turned out to be false. None of that is about restricting AI use so much as making sure the thinking still happens somewhere, even when the typing doesn’t have to.

The Metrics Problem

Here’s where I think most engineering organisations are currently getting this wrong. We still mostly measure junior engineers the way we always did: tickets closed, pull requests merged, velocity contributed. Those were always rough proxies for judgement, not judgement itself, but they used to correlate well enough that nobody worried about the gap. The UNSW numbers suggest the gap has opened up. A junior engineer can now produce more, faster, while understanding and retaining less of what they produced, and on the old metrics that looks like a star performer.

If a junior engineer ships code they can’t explain in a review, I think that’s a real problem now, because somebody still has to own that code in six months, and a bigger problem later, because that junior engineer is supposed to become the senior engineer who can diagnose a production incident at 2am without relying completely on a model. Adrian Harwood’s phrase for the underlying skill is “distrusting everything that you produce,” whether a human or an AI wrote it. That’s a habit, and habits get built through repetition under supervision, not through a tool that happily produces a plausible answer on the first try.

What to Do Next

I think that it’s time to stop treating “how much did the junior ship” as a meaningful signal on its own and start asking for evidence of the reasoning behind it, in the review itself rather than in a separate process nobody has time for. That means reviewers asking “why this approach and not the obvious alternative” as a standard comment, not an occasional one. It means junior engineers being expected to walk a reviewer through code they didn’t fully write themselves, the same way they’d be expected to walk through code a human teammate handed them. And it means being honest in performance conversations that shipping less, or deciding not to ship a ticket at all because the assumptions didn’t hold up, can be the stronger signal, not the weaker one.

I don’t think this is an argument against using AI tools for junior engineers, and I don’t think either LeadDev piece makes that argument either; I think it’s an argument against assuming the tools will sort out judgement on their own, the way we quietly assumed debugging and code review used to sort it out on their own. If you don’t deliberately build the friction back in somewhere, you’ll get engineers who look more productive every quarter and less capable every year, and you won’t notice until the senior engineers who were supposed to replace the current ones simply aren’t ready when you need them.