Table of Contents

Introduction

  1. Problems in NLP:
    • Question Answering (QA)
    • Information Extraction (IE)
    • Sentiment Analysis
    • Machine Translation (MT)
    • Spam Detection
    • Parts-of-Speech (POS) Tagging
    • Named Entity Recognition (NER)
    • Coreference Resolution
    • Word Sense Disambiguation (WSD)
    • Parsing
    • Paraphrasing
    • Summarization
    • Dialog
  2. (mostly) Solved Problems in NLP:
    • Spam Detection
    • Parts-of-Speech (POS) Tagging
    • Named Entity Recognition (NER)
  3. Within-Reach Problems:
    • Sentiment Analysis
    • Coreference Resolution
    • Word Sense Disambiguation (WSD)
    • Parsing
    • Machine Translation (MT)
    • Information Extraction (IE)
  4. Open Problems in NLP:
    • Question Answering (QA)
    • Paraphrasing
    • Summarization
    • Dialog
  5. Issues in NLP (why nlp is hard?):
    • Non-Standard English: “Great Job @ahmed_badary! I luv u 2!! were SOO PROUD of dis.”
    • Segmentation Issues: “New York-New Haven” vs “New-York New-Haven”
    • Idioms: “dark horse”, “getting cold feet”, “losing face”
    • Neologisms: “unfriend”, “retweet”, “google”, “bromance”
    • World Knowledge: “Ahmed and Zach are brothers”, “Ahmed and Zach are fathers”
    • Tricky Entity Names: “Where is Life of Pie playing tonight?”, “Let it be was a hit song!”
  6. Tools we need for NLP:
    • Knowledge about Language.
    • Knowledge about the World.
    • A way to combine knowledge sources.
  7. Methods:
    In general we need to construct Probabilistic Models built from language data.
    We do so by using rough text features.

    All the names models, methods, and tools mentioned above will be introduced later as you progress in the text.

Definitions

  1. K-Nearest-Neighbors:
    Complexity:
    • Training: \(\:\:\:\:\mathcal{O}(1)\)
    • Predict: \(\:\:\:\:\mathcal{O}(N)\)

  2. Asynchronous:

  3. Asynchronous:

  4. Asynchronous:

  5. Asynchronous:

  6. Asynchronous:

  7. Asynchronous:

  8. Asynchronous:

Metrics

  1. L1 Distance:
    \[d_1(I_1, I_2) = \sum_p{\|I_1^p - I_2^p\|}\]
    Pixel-wise absolute value differences.
  2. L2 Distance:
    \[d_2 (I_1, I_2) = \sqrt{\sum_{p} \left( I^p_1 - I^p_2 \right)^2}\]
  3. L1 vs. L2:
    The L2 distance penalizes errors (pixel differences) much more than the L1 metric does.
    The L2 distnace will be small iff there are man small differences in the two vectors but will explode if there is even one big difference between them.
    Another difference we highlight is that the L1 distance is dependent on the corrdinate system frame, while the L2 distance is coordinate-invariant.

  4. Asynchronous:

  5. Asynchronous:

  6. Asynchronous:

  7. Asynchronous:

  8. Asynchronous: