MPCS 53113 Natural Language Processing (Summer 2022)

Section 1
Instructor(s) Chaudhary, Amitabh (amitabh)
Location RY 251 (In-Person or Online)
Meeting Times Wednesday 5:30pm - 8:30pm
Fulfills Elective Specialization - Data Analytics (DA-2)

Syllabus

*Please note: This is the syllabus from the 2021/22 academic year and subject to change.*

Natural language processing (NLP) is the application of computational techniques, particularly from machine learning, to analyze and synthesize  human language. The recent explosion in the amount of available text data has made natural language processing invaluable for businesses, social sciences, and even natural sciences.
 
In this course we study the fundamentals of modern natural language processing, emphasizing models based on deep learning.   These include language models,  word embeddings,  recurrent neural networks (Simple RNNs, LSTMs), context-free grammars and syntactic parsing, dependency parsing, and attention-based models such as the transformer and BERT.
 
We use Python and Python based libraries such as PyTorch, NLTK, and SpaCy for implementing algorithms and processing text.
 
A significant component is the course project in which students apply NLP techniques to solve a real-world problem.
 


Topics
A tentative list of topics follows.

  • Language Models
  • Word Embeddings
  • Recurrent Neural Networks (RNNs), LSTMs for NLP
  • Convolutional Neural Networks (CNNs)  for NLP
  • Conditioned Generation, Sequence to Sequence Models
  • Attention-Based Models, The Transformer
  • Sytactic Parsing, Context Free Grammars
  • BERT 

Coursework and Evaluation

  • Assignments:   The assignments will be approximately one every week, for the the first half of the course. (In the second half, there will be fewer assignments to allow students to focus on their projects.)  They are designed to reinforce material and test a deeper understanding of the concepts and algorithms through theoretical questions, program implementation, and analysis of empirical results.  Tentatively worth 40% of the grade.
  • Midterm Examination: Tentatively worth 20% of the grade.
  • Course Project: Tentatively worth 40% of the grade.

Textbook

  • Neural Network Methods for Natural Language Processing by Yoav Goldberg (https://doi.org/10.2200/S00762ED1V01Y201703HLT037) 

Course Prerequisites

A grade of B+ or better in the following courses:
• MPCS 50103 Math for Computer Science (or placement exam waiver)
• MPCS 51042 Python Programming

A grade of B or better in MPCS 55001 Algorithms

A grade of B or better in one of the following courses:
• MPCS 53110 Foundations of Computational Data Analysis
• MPCS 53120 Applied Data Analysis

MPCS 53111 Machine Learning (recommended; see below).
Equivalent courses or experience will be accepted with instructor permission. A prior course in machine learning would be useful but is not necessary; if you haven't taken any please contact the instructor with your prior courses and experience.

Other Prerequisites

Programming experience in Python.

For MPCS students, this is an elective/specialization course and can only be taken after completion of three core classes or concurrent with third core class.

Overlapping Classes

This class is scheduled at a time that conflicts with these other classes:

  • MPCS 50103-1 -- Mathematics for Computer Science: Discrete Mathematics

Eligible Programs

Masters Program in Computer Science