Introduction

1. Problems in NLP:

   • Question Answering (QA)
   • Information Extraction (IE)
   • Sentiment Analysis
   • Machine Translation (MT)
   • Spam Detection
   • Parts-of-Speech (POS) Tagging
   • Named Entity Recognition (NER)
   • Coreference Resolution
   • Word Sense Disambiguation (WSD)
   • Parsing
   • Paraphrasing
   • Summarization
   • Dialog

2. (mostly) Solved Problems in NLP:

   • Spam Detection
   • Parts-of-Speech (POS) Tagging
   • Named Entity Recognition (NER)

3. Within-Reach Problems:

   • Sentiment Analysis
   • Coreference Resolution
   • Word Sense Disambiguation (WSD)
   • Parsing
   • Machine Translation (MT)
   • Information Extraction (IE)

4. Open Problems in NLP:

   • Question Answering (QA)
   • Paraphrasing
   • Summarization
   • Dialog

5. Issues in NLP (why nlp is hard?):

   • Non-Standard English: “Great Job @ahmed_badary! I luv u 2!! were SOO PROUD of dis.”
   • Segmentation Issues: “New York-New Haven” vs “New-York New-Haven”
   • Idioms: “dark horse”, “getting cold feet”, “losing face”
   • Neologisms: “unfriend”, “retweet”, “google”, “bromance”
   • World Knowledge: “Ahmed and Zach are brothers”, “Ahmed and Zach are fathers”
   • Tricky Entity Names: “Where is Life of Pie playing tonight?”, “Let it be was a hit song!”

6. Tools we need for NLP:

   • Knowledge about Language.
   • Knowledge about the World.
   • A way to combine knowledge sources.

7. Methods:

   In general we need to construct Probabilistic Models built from language data.

   We do so by using rough text features.

   All the concepts, models, methods, and tools mentioned above will be introduced later as you progress in the text.