Definition
Natural Language Processing (NLP) is the field of AI that enables computers to read, understand and produce human language. It has two sides: understanding language (NLU) and generating it (NLG).
In simple words
When your phone suggests the next word, when Gmail flags a phishing mail, when Google translates Hindi to English: that's NLP. It turns messy human language into something a computer can work with, and back again.
Why it matters
Most of the world's knowledge is in text and speech. NLP powers search, translation, voice assistants, document analysis and LLMs. For India, with its many languages and its code-mixed Hinglish, it's especially important.
How it works
Why language is hard
- Ambiguity: "bank" may be a river bank or a money bank.
- Context: "It's cold" could be weather, food or a person's manner.
- Sarcasm: "Great, another Monday" isn't praise.
- Variety: slang, spelling mistakes, and mixing languages.
A typical pipeline
- Clean the text and split it into tokens.
- Turn the tokens into numbers, using TF-IDF or embeddings.
- Train a model, or use a pre-trained one.
- Produce the output: a label, an answer or generated text.
| Task | Example |
|---|---|
| Classification | Is this review positive? |
| Named entity recognition | Find people, places and dates in text |
| Translation | Hindi to English |
| Summarisation | A long report into five lines |
| Question answering | Answer from a document |
Three eras
NLP began with hand-written rules. From the 1990s statistical methods took over, and from the 2010s neural networks did. Since 2018 Transformers have led most of the field.
Try it yourself
Inside a translation app
Follow one Hindi sentence on its way to English.
Why would an app get confused?
Each sentence is hard for a computer. Find the reason.
"The bat flew out of my hand."
"gr8 c u tmrw"
"I sat on the bank."
"Kal milte hain, ok?"
"Wonderful, my phone died again."
"Great, another Monday."
Understanding or generating?
NLP has two sides. Is each app understanding language, or producing it?
Writing a birthday message for you
Drafting a reply to a customer
Flagging a phishing email
Finding the dates in a long email
Common mistakes
- Assuming a model trained on English works on Hindi or Hinglish. Test on your own language.
- Removing stop words automatically with modern Transformers. They use those words for context.
Check yourself
Summary
- NLP is how computers read and produce human language.
- Ambiguity and context make it hard.
- It moved from rules, to statistics, to neural networks and Transformers.
Finished reading?