Definition
AI systems fail in predictable ways: bias from unfair data, hallucination in generative models, brittleness on inputs unlike their training data, and over-trust by the people using them. These are limits of how the systems work, and they don't go away just because a model seems confident.
In simple words
A student who only studied last year's question paper can score well on it, and fail on a new one. AI is similar: it's good at what looks like its training data. It's also often unable to say "I don't know", so it may give a wrong answer in the same confident tone.
Why it matters
These failures have real costs. A resume filter can reject qualified people, a medical model can miss patients, and a lawyer can cite a case that doesn't exist. Knowing the failure types is the first step to using AI safely.
How it works
| Failure | What happens | Real example | What helps |
|---|---|---|---|
| Bias | Unfair patterns in the data are learned | Gender Shades (2018): commercial face analysis erred up to 34.7% for darker-skinned women, and up to 0.8% for lighter-skinned men | Better data, testing per group |
| Hallucination | Confident, made-up facts | Fake court cases cited in legal filings | Checking sources, RAG |
| Brittleness | Breaks on unfamiliar input | Small changes to an image can fool a vision model | Testing on varied data, monitoring |
| Data drift | The world changes after training | Demand models after COVID-19 began | Monitoring, retraining |
| Over-trust | People stop checking | Accepting wrong suggestions without review | A human review step |
Common mistakes
- Taking AI output as fact when it matters. Check it, as you would check an unknown website.
- Assuming a model is fair because no one programmed bias into it. Bias usually arrives through the data.
- Testing only on data similar to the training set.
Check yourself
Summary
- Main failures: bias, hallucination, brittleness, drift, over-trust.
- Confidence doesn't mean correctness.
- Test per group, monitor after launch, and keep a person in the loop for important decisions.
Finished reading?