Definition
The deep learning boom began in 2012, when a neural network called AlexNet won the ImageNet competition by a wide margin. It was followed by AlphaGo beating top Go players in 2016, and the Transformer architecture in 2017, which later powered LLMs.
In simple words
Three things came together: huge datasets, powerful GPUs, and better training tricks. An idea that had waited for decades suddenly worked, and within a few years nearly everyone in the field had switched to it.
Why it matters
It explains the AI you see today, and shows that a breakthrough can depend on data and hardware as much as on a new theory.
How it works
| Year | Event | Why it mattered |
|---|---|---|
| 2012 | AlexNet | Far ahead of other methods at image recognition |
| 2014 | GANs | Networks that create realistic images |
| 2015 | ResNet | Very deep networks became trainable |
| 2016 | AlphaGo | Beat Lee Sedol at Go, using deep RL and tree search |
| 2017 | Transformer | The base of today's LLMs |
| 2020 | AlphaFold 2 | Major step in predicting protein structures |
Try it yourself
How we got from pictures to chatbots
Three things came together, and then things moved fast.
Build the AI timeline to today
Put these breakthroughs in order, earliest first.
- 1Researchers publish the Transformer, the design behind today's chatbots
- 2GPT-3 shows that very large language models can write fluently
- 3AlphaGo beats Lee Sedol, a world champion at Go
- 4A deep neural network wins the ImageNet image contest
- 5ChatGPT is released and reaches millions of people within weeks
Common mistakes
- Thinking neural networks were invented in 2012. They're decades older. 2012 is when they clearly won at scale.
- Crediting only the algorithm. Data and GPUs were equally crucial.
Check yourself
Summary
- Data, GPUs and better methods came together around 2012.
- AlphaGo and the Transformer followed.
Finished reading?
Sources: Krizhevsky et al., AlexNet (2012) · Vaswani et al. (2017)