AI keeps taking on more of the work

In 2012 a neural network called AlexNet won the ImageNet competition by a margin that made people who had spent their careers hand-tuning features go a bit quiet. It ran on two gaming GPUs. I remember reading about it and thinking, fine, image recognition, nice. I didn’t think it would lead here.

This is what the last fourteen years look like from where I’m sitting. It’s my version, so the dates are the years I noticed things, and a researcher would draw it differently.

The deep learning years

2013 brought word2vec, which turned words into vectors you could do arithmetic on. King minus man plus woman equals queen. It was a party trick, but it was also the first time I saw a machine hold something like meaning in a list of numbers.

2014 was seq2seq and attention. Two networks, one reading a sentence and one writing it out in another language, with attention letting the writer peek back at the right words while it wrote. That trick, the peeking, is the one that ended up mattering most.

2015 was ResNet, which made training very deep networks much easier. Then in March 2016 AlphaGo beat Lee Sedol at Go. I watched some of those games. Move 37 in game two still gets talked about, and I think that match was the first time a lot of people outside the field paid attention.

Through all of this the model was the product. You collected labelled data for one task, trained a network for one task, and shipped one task. Cats or not cats. This language into that one.

The transformer years

June 2017, a paper called “Attention Is All You Need”. It threw away the recurrent part and kept only the peeking. The Transformer. Honestly, at the time it read like a translation paper with a cheeky title.

2018 gave us BERT and GPT-1. Same building block, two directions. BERT read text both ways and got very good at understanding it; GPT read left to right and got okay at writing it. 2019 was GPT-2, which OpenAI held back at first because they worried about what it could write. That felt dramatic then. It was 1.5 billion parameters.

2020 was GPT-3, 175 billion parameters, and this is where it stopped being a lab thing for me. You gave it a few examples in the prompt and it did the task. No task-specific training. The same model wrote poems, SQL, apologies and cover letters. I started to question whether each task needed its own model.

2021, Codex and the GitHub Copilot preview. Autocomplete that finished the whole function. I was suspicious of it at first, mostly because it was right just often enough to make you lazy.

The chat years

30 November 2022, ChatGPT. Not much new under the hood compared with what was already available through the API, and yet the chat box changed who used it. People who had never cared about any of this started trying it.

2023 was GPT-4 in March, and Meta’s Llama, which leaked and then got a proper open release with Llama 2 and brought more attention to open weights. Suddenly you could run a decent model on your own machine. People did, on laptops that got very hot.

The agent years

2024 and 2025 were about two things. Reasoning models, which spend more computation on an answer and improved on maths and code. And agents, which is a boring word for a model in a loop with tools: read a file, run a command, look at the result, try again.

That loop is the whole story of 2026 as far as I’m concerned. Coding agents like Claude Code and Codex (OpenAI reused the name) sit in your terminal, take a task, and go. They read the repo, make the change, run the tests, fix what broke. Not perfectly. But the shape of the work changed: I describe it, the agent does it, I read what it did, and sometimes I send it back.

Some opinions

What I take from the timeline is that every few years the unit of work got bigger. In 2012 the unit was a label. In 2017 it was a sentence. In 2020 it was a prompt. In 2026 it’s a task, and the task can take an hour.

The other thing: attention, the peeking trick from 2014, still sits inside the models I use. Fourteen years and the main idea is over a decade old. The scale still surprises me.

What I got wrong, for the record. I thought the chat interface was a gimmick in 2022 and I thought agents would take a lot longer than they did. I still think the word “intelligence” is doing work it hasn’t earned.