What Is Machine Learning?

Definition

Nobody wrote a rule for every spam email. So how does your inbox catch them?

Machine Learning (ML) is the part of AI in which a computer learns patterns from data, instead of being given step-by-step rules. Tom Mitchell's definition (1997) is the standard one: a program learns from experience E, for tasks T and performance measure P, if its performance at T, measured by P, improves with E.

In simple words

Think of teaching a child what a mango is. You don't give a list of rules about colour and shape. You point at mangoes and say "mango". After enough examples, the child recognises a new mango. ML works the same way: show many examples with the right answers, and the computer works out the pattern by itself.

Why it matters

Some problems are too hard to write as rules. Try writing rules that recognise any handwritten 7, or any fraudulent payment. ML solves these by learning from examples, and it keeps improving as more data arrives. It powers recommendations, fraud checks, voice typing, translation and chatbots.

How it works

Programming vs machine learning

	You give the computer	You get back
Traditional programming	Rules + data	Answers
Machine learning	Data + answers	Rules (a model)

Mitchell's definition, on a spam filter

Task T: decide whether an email is spam.
Experience E: thousands of emails that people have already marked.
Performance P: the share of new emails it gets right.

The steps of learning

Collect examples, each with its features (what you know) and label (the right answer).
Choose a model: a formula with adjustable numbers called parameters.
Train: the model predicts, measures how wrong it was, and adjusts its parameters. Repeat many times.
Test on examples it has never seen. That is the only fair measure.
Use it on new data. This step is called inference.

The three main kinds

Kind	Learns from	Example
Supervised	Examples with right answers	Predicting house prices
Unsupervised	Examples with no answers	Grouping similar customers
Reinforcement	Rewards for its actions	Learning to play a game

The maths

What every learning algorithm does

θ^{*} = ar g θ min \frac{1}{n} i = 1 \sum n L (f_{θ} (x_{i}), y_{i})

$θ$: The model's parameters, the numbers it can adjust
$θ^{*}$: The best parameters found
$f_{θ} (x_{i})$: The model's prediction for example i
$y_{i}$: The right answer for example i
$L$: The loss: how wrong one prediction is
$n$: The number of examples

In words: find the parameters that make the average error as small as possible. Suppose a model predicts 3 house prices (in lakhs) as 50, 60 and 80, and the right answers are 52, 60 and 75. Using squared error as the loss:

Average loss = [ (50−52)² + (60−60)² + (80−75)² ] / 3
             = [ 4 + 0 + 25 ] / 3
             = 9.67

Training adjusts the parameters so this number falls. That's all "learning" means here.

In code

A complete ML program with scikit-learn. The model is never told the rule behind the numbers. It works it out from the examples.

first_model.py

from sklearn.linear_model import LinearRegression

# Experience: house size in sq ft, and price in lakhs
sizes = [[500], [800], [1000], [1200], [1500]]
prices = [25, 40, 50, 60, 75]

model = LinearRegression()
model.fit(sizes, prices)          # learning happens here

print("Learned rule: price =", round(model.coef_[0], 3), "x size +", round(model.intercept_, 3))
print("A 1100 sq ft house:", round(model.predict([[1100]])[0], 1), "lakhs")

Output

Learned rule: price = 0.05 x size + 0.0
A 1100 sq ft house: 55.0 lakhs

Common mistakes

Testing a model on the data it trained on. It may have just memorised the answers.
Thinking more data always fixes everything. Wrong or biased data teaches wrong patterns.
Using ML when a simple rule would do. If you can write the rule in five lines, write the rule.

Check yourself

1. In traditional programming you supply rules and data. In ML you supply:

2. For a spam filter, what is the experience E in Mitchell's definition?

3. Grouping customers with no labels is which kind of ML?

Summary

ML learns patterns from data, instead of following hand-written rules.
Mitchell: performance at a task, measured by P, improves with experience E.
Training means adjusting parameters to reduce the average loss.
Three main kinds: supervised, unsupervised and reinforcement.

Read it. Now try it yourself.

Open the interactive activity

Finished reading?

Sources: Mitchell, Machine Learning (1997) · Google Machine Learning Crash Course