Definition
MLOps is a set of practices for reliably building, deploying, monitoring and updating machine learning models in production. It applies DevOps ideas such as automation and testing to ML, and adds what's special to it: data and model versioning, drift monitoring and retraining.
In simple words
A notebook model is like a home-cooked dish. MLOps is everything needed to serve that same dish in a chain of restaurants, every day, at the same quality: written recipes, trained staff, quality checks, and changing the menu when tastes change.
Why it matters
Most ML value is lost after the notebook. Models silently decay as the world changes, and unlike normal software a model can keep running while giving worse and worse answers. MLOps is how you notice and fix that.
How it works
| Stage | What's done | Example tools |
|---|---|---|
| Data | Validate and version it | DVC, Great Expectations |
| Experiment | Track every run and its results | MLflow, Weights & Biases |
| Package | Save the model with its environment | Docker, MLflow registry |
| Deploy | Serve it behind an API | FastAPI, Kubernetes |
| Monitor | Watch latency, errors, data and quality | Prometheus, Evidently |
| Retrain | Update it when it degrades | Airflow, CI/CD |
Two kinds of drift
- Data drift: the inputs change. For example, more users start coming from phones.
- Concept drift: the link between input and answer changes. For example, fraudsters adopt a new trick.
In code
One simple drift check: compare an input's distribution at training time with its distribution today.
import numpy as np
rng = np.random.default_rng(0)
training_ages = rng.normal(35, 8, 5000)
todays_ages = rng.normal(27, 6, 5000)
def psi(expected, actual, bins=10):
edges = np.quantile(expected, np.linspace(0, 1, bins + 1))
edges[0], edges[-1] = -np.inf, np.inf
e = np.histogram(expected, edges)[0] / len(expected)
a = np.histogram(actual, edges)[0] / len(actual)
a = np.clip(a, 1e-6, None)
return np.sum((a - e) * np.log(a / e))
print("Same data PSI:", round(psi(training_ages, rng.normal(35, 8, 5000)), 3))
print("Drifted data PSI:", round(psi(training_ages, todays_ages), 3))
print("A common rule of thumb: above 0.25 means a big shift.")Output
Same data PSI: 0.002 Drifted data PSI: 1.469 A common rule of thumb: above 0.25 means a big shift.
Common mistakes
- Treating deployment as the finish line. It's where the real work starts.
- Monitoring only server health. A model can be up, fast and wrong.
- Being unable to reproduce a model because data and code versions weren't saved.
Check yourself
Summary
- MLOps keeps models working reliably in production.
- It's DevOps, plus data, model versioning and drift.
- A model can be running and wrong, so monitor quality as well as uptime.
Finished reading?
Sources: Google Cloud: MLOps