Posts Tagged "AI"

A study from the University of New South Wales this year gave undergraduate computer science students three introductory C programming tasks. Half the group used ChatGPT-4.5, half used ordinary web search. The AI-assisted group scored 89% on the tasks against 69% for the web-search group, a 20-point gap. Two wo days later they took a retention test which showed that the ChatGPT group scored 39%, the web-search group 52%. Asked how much of their submitted code was really theirs, the ChatGPT group put it at 45%. The web-search group put it at 81%. I think that's the number worth thinking about, not the first one.

I read something last week that unsettled me more than most of the AI articles that fill my inbox, and it wasn't about job losses or productivity charts. It was about motivation: why would anyone bother getting good at something, in public, if the thing they're good at is about to become invisible?

Fiona Chen and James Stratton, two economists at Harvard, have written a paper that I think anybody leading an engineering team should read. It uses work data from 718 companies to ask what happened to software output and jobs once AI coding tools arrived, and what they found is that people wrote a lot more code without a matching increase in software shipped.

Annie Vella and Kelly Blincoe published a longitudinal study of professional software engineers using AI coding assistants, and it turned up this week in one of the engineering-leadership newsletters I read. The headline number is the one everyone already half-believed: 82% of the developers they surveyed said they were spending less time writing code six months after they'd started using AI assistants regularly. What I think is more interesting, and more useful if you're trying to lead an engineering org through this, is what happened underneath that number.

David Heinemeier Hansson stood up at Rails World last week and told a room full of programmers that he's retired. Not from 37signals, the company he still runs. From programming itself. "I spent a quarter of a damn century chiseling code by hand and loving every moment of it," he said, "and this is something to look back upon with joy and accept that it is over." As Gergely Orosz reported in [The Pragmatic Engineer](https://newsletter.pragmaticengineer.com/p/the-pulse-end-of-coding-by-hand), the internet did what the internet does with a line like that: it took the seven words that fit in a headline and ran with them, past the part of the same keynote that actually matters for anyone leading a team through this.

Will Larson published something last week that's been doing the rounds in engineering leadership circles: a write-up of what he's calling the "software factory pattern," an experiment he's running at Imprint, the fintech company where he's CTO. I think most people are reading it slightly wrong, including some of the people criticising it.