Blog Archive

A study from the University of New South Wales this year gave undergraduate computer science students three introductory C programming tasks. Half the group used ChatGPT-4.5, half used ordinary web search. The AI-assisted group scored 89% on the tasks against 69% for the web-search group, a 20-point gap. Two wo days later they took a retention test which showed that the ChatGPT group scored 39%, the web-search group 52%. Asked how much of their submitted code was really theirs, the ChatGPT group put it at 45%. The web-search group put it at 81%. I think that's the number worth thinking about, not the first one.

I read something last week that unsettled me more than most of the AI articles that fill my inbox, and it wasn't about job losses or productivity charts. It was about motivation: why would anyone bother getting good at something, in public, if the thing they're good at is about to become invisible?

Fiona Chen and James Stratton, two economists at Harvard, have written a paper that I think anybody leading an engineering team should read. It uses work data from 718 companies to ask what happened to software output and jobs once AI coding tools arrived, and what they found is that people wrote a lot more code without a matching increase in software shipped.

Annie Vella and Kelly Blincoe published a longitudinal study of professional software engineers using AI coding assistants, and it turned up this week in one of the engineering-leadership newsletters I read. The headline number is the one everyone already half-believed: 82% of the developers they surveyed said they were spending less time writing code six months after they'd started using AI assistants regularly. What I think is more interesting, and more useful if you're trying to lead an engineering org through this, is what happened underneath that number.

What I've read this week all point to the same, somewhat unglamorous, truth: leading an engineering team well has very little to do with technical brilliance and almost everything to do with the daily discipline of decisions, honesty and attention. [[real example needed: a specific moment from Stephen's own leadership experience — a decision he made as VP Engineering that rippled onto his team, a management archetype he recognises in himself, or a postmortem where the team asked "what are we changing" rather than "why did this happen"]] Eight pieces came through the inbox this week that, between them, map out what that discipline actually looks like in practice.

David Heinemeier Hansson stood up at Rails World last week and told a room full of programmers that he's retired. Not from 37signals, the company he still runs. From programming itself. "I spent a quarter of a damn century chiseling code by hand and loving every moment of it," he said, "and this is something to look back upon with joy and accept that it is over." As Gergely Orosz reported in [The Pragmatic Engineer](https://newsletter.pragmaticengineer.com/p/the-pulse-end-of-coding-by-hand), the internet did what the internet does with a line like that: it took the seven words that fit in a headline and ran with them, past the part of the same keynote that actually matters for anyone leading a team through this.

Will Larson published something last week that's been doing the rounds in engineering leadership circles: a write-up of what he's calling the "software factory pattern," an experiment he's running at Imprint, the fintech company where he's CTO. I think most people are reading it slightly wrong, including some of the people criticising it.

Having your team all working towards a common set of goals is extremely important. However, as leaders and managers we can sometimes get far too hung up on what needs to be delivered and forget about who is actually doing the work. Forgetting about the people and focusing only on delivery goals may get you to one particular deadline but failing to build the team effectively and align them around a common set of principles will rapidly cause longer term problems. A team charter can help prevent this.