The Machine Learning Workflow

Definition

In simple words

Building a model is like cooking for guests. Decide the menu, buy ingredients, clean and chop, cook, taste, serve, and ask how it was. Most of the work is the prep. Beginners imagine that ML is all cooking, and then find that it's mostly chopping.

Why it matters

Many ML projects fail for reasons that have nothing to do with algorithms: the wrong question, bad data, or no plan for after launch. Following the workflow makes those failures visible early.

How it works

1. Define the problem. What decision will this change, and how will you measure success? Is ML even needed?
2. Get the data. Check where it comes from, and whether you may use it.
3. Explore it (EDA) with charts and summaries.
4. Prepare it: clean, encode, scale and split.
5. Set a baseline, the simplest possible answer, so you know what to beat.
6. Train and compare models.
7. Evaluate on the test set, once, at the end.
8. Deploy it, usually behind an API.
9. Monitor it, and retrain when it degrades.

In code

The whole workflow in a few lines, including the baseline and a final test.

from sklearn.datasets import load_breast_cancer
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0, stratify=y)

baseline = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X_train, y_train)

print("Baseline accuracy:", round(baseline.score(X_test, y_test), 3))
print("Model accuracy:", round(model.score(X_test, y_test), 3))

Baseline accuracy: 0.632
Model accuracy: 0.982

Common mistakes

• Jumping to the model before understanding the problem and data.
• Reusing the test set while tuning. Then it's no longer a fair exam.
• Skipping the baseline, so you can't tell whether the model helps at all.

Check yourself

Summary

• Problem, data, explore, prepare, baseline, train, evaluate, deploy, monitor.
• It's a cycle, and the prep takes most of the time.
• Beat a baseline, and touch the test set only once.