Definition
Why did AI suddenly get so good at seeing and talking after 2012?
Deep Learning is a branch of machine learning that uses neural networks with many layers. Each layer learns to turn its input into a slightly more useful representation, so the network works out the important features for itself.
In simple words
To recognise a cat in a photo, the first layers notice edges. The next layers combine edges into shapes like eyes and ears. Later layers combine those into a whole cat. Nobody tells the network what to look for. It's like an assembly line, with each worker building on the work of the last.
Why it matters
Deep learning made today's speech recognition, translation, image generation, protein structure prediction and LLMs possible. Classic ML needs people to hand-craft features, while deep learning learns them, so it can handle raw images, audio and text.
How it works
Why does it work now?
The core idea is decades old. Three things changed around 2012, when a network called AlexNet won the ImageNet competition by a wide margin.
- Data: the internet produced huge labelled datasets.
- Compute: GPUs made training many times faster.
- Algorithms: ReLU, dropout, better initialisation and later Transformers made deep networks trainable.
Deep learning vs classic ML
| Classic ML | Deep learning | |
|---|---|---|
| Features | Usually made by people | Learned from the raw data |
| Data needed | Works with less | Usually needs a lot |
| Hardware | A normal CPU is often enough | GPUs for anything large |
| Best on | Tables | Images, audio, text |
| Explaining it | Often easier | Hard |
In code
A small network learns XOR, a pattern no straight line can separate. A hidden layer is what makes this possible.
from sklearn.neural_network import MLPClassifier
X = [[0, 0], [0, 1], [1, 0], [1, 1]]
y = [0, 1, 1, 0] # XOR
network = MLPClassifier(hidden_layer_sizes=(8,), activation="tanh", solver="lbfgs", max_iter=2000, random_state=1)
network.fit(X, y)
print("Layers:", network.n_layers_)
print("Predictions:", network.predict(X).tolist())
print("Correct: ", y)Output
Layers: 3 Predictions: [0, 1, 1, 0] Correct: [0, 1, 1, 0]
Common mistakes
- Reaching for deep learning first. On a small table of data it's often slower, costlier and no better.
- Thinking neural networks work like the brain. The inspiration is loose.
- Treating the model as magic. Its predictions are only as good as its data.
Check yourself
Summary
- Deep learning is ML with many-layered neural networks.
- It learns its own features, layer by layer.
- Data, GPUs and better algorithms made it take off after 2012.
Finished reading?