||*This course will be conducted remotely and will be online only for Summer 2021*
Natural language processing (NLP) is the application of computational techniques, particularly from machine learning, to analyze and synthesize human language. The recent explosion in the amount of available text data has made natural language processing invaluable for businesses, social sciences, and even natural sciences.
In this course we study the fundamentals of modern natural language processing, emphasizing models based on deep learning. These include language models, word embeddings, recurrent neural networks (Simple RNNs, LSTMs), context-free grammars and syntactic parsing, dependency parsing, and attention-based models such as the transformer and BERT.
We use Python and Python based libraries such as PyTorch, NLTK, and SpaCy for implementing algorithms and processing text.
A significant component is the course project in which students apply NLP techniques to solve a real-world problem.
A tentative list of topics follows.
Coursework and Evaluation
- Language Models
- Word Embeddings
- Recurrent Neural Networks (RNNs), LSTMs for NLP
- Convolutional Neural Networks (CNNs) for NLP
- Conditioned Generation, Sequence to Sequence Models
- Attention-Based Models, The Transformer
- Sytactic Parsing, Context Free Grammars
- Assignments: The assignments will be approximately one every week, for the the first half of the course. (In the second half, there will be fewer assignments to allow students to focus on their projects.) They are designed to reinforce material and test a deeper understanding of the concepts and algorithms through theoretical questions, program implementation, and analysis of empirical results. Tentatively worth 40% of the grade.
- Midterm Examination: Tentatively worth 20% of the grade.
- Course Project: Tentatively worth 40% of the grade.
- Neural Network Methods for Natural Language Processing by Yoav Goldberg (https://doi.org/10.2200/S00762ED1V01Y201703HLT037)
A grade of B+ or better in the following courses:
• MPCS 50103 Math for Computer Science (or placement exam waiver)
• MPCS 51042 Python Programming
A grade of B or better in MPCS 55001 Algorithms
A grade of B or better in one of the following courses:
• MPCS 53110 Foundations of Computational Data Analysis
• MPCS 53120 Applied Data Analysis
MPCS 53111 Machine Learning (recommended; see below).
Equivalent courses or experience will be accepted with instructor permission. A prior course in machine learning would be useful but is not necessary; if you haven't taken any please contact the instructor with your prior courses and experience.