MPCS 57200 Generative AI (Winter 2026)

Overview

Recent generative AI systems such as ChatGPT, Dall-E, and GitHub Copilot have shown its applicability to a wide range of problem domains, and it is quickly becoming a valuable tool in the software developer's toolbox.

This course will provide an experienced software developer with no experience in Machine Learning with

• a conceptual understanding of the basic concepts of Generative AI
• how to build Generative AI systems to generate output targeting their domain of interest
• how to deploy and integrate Generative AI models into production applications

Concepts

• History of Generative AI

• Neural network concepts

  • The Fundamental Equations of Deep Learning
  • Activation functions
  • Training with gradient descent
  • Key mathematical concepts like cross-entropy, softmax, and perplexity and their role in Deep Learning Generative AI
  • Scaling laws

• Generative AI concepts

  • Encoder/decoder architectures as basis for Generative AI
  • The role of the latent space
  • Generative Adversarial Nets
  • Transformer architectures and Attention
  • Large Language Models

• Customizing Generative AI

  • While most engineers will not build their own production models from scratch (which typically costs tens of millions of dollars), they will need to customize them for their application
  • enhanced pre-training
  • fine-tuning
  • Prompt engineering and few-shot learning

• Strengths and Weaknesses: LLMs are not a silver bullet

  • Can LLMs reason?
  • Responsible AI

• Incorporating Generative AI into applications

  • Model deployment
  • Langchain
  • Thinking beyond chatbots: non-generative uses of generative AI

Labs

• Generating text programmatically with an LLM
• Image generation with the fashion MNIST database
  • With an auto-encoder
  • Improve with a variational autoencoder
• Aging and de-aging faces
• Building and training a very simple LLM from scratch
• Fine-tuning a model
• Prompt Engineering playground
• Responsible AI labs
• Generative Adversarial Nets
• Langchain
• Deploying a trained model for use in production

Recommended text

While we don't closely follow any textbook, a number of the labs and explanations follow David Foster's excellent O'Reilly book Generative Deep Learning, 2nd Edition (If you purchase this, make sure you get the 2nd edition). We will also look at a variety of seminal and topical papers.

B+ or above in MPCS 51036, 51040, 51042, 51046, 51100, or have a Core Waiver for Programming.

The main prerequisite is a basic familiarity with Python. We do not assume any background in machine learning or statistics.

This course requires competency in Unix and Linux. If you attended the MPCS Unix Bootcamp you covered the required material. If you did not, please review the UChicago CS Student Resource Guide here: https://uchicago-cs.github.io/student-resource-guide/.

Course request information for non-MPCS students: https://masters.cs.uchicago.edu/student-resources/non-mpcs-student-course-requests/

This class is scheduled at a time that conflicts with these other classes:

• MPCS 55001-1 -- Algorithms
• MPCS 51082-1 -- Introduction to Unix Systems
• MPCS 52011-1 -- Introduction to Computer Systems
• MPCS 51230-1 -- User Interface and User Experience Design
• MPCS 53001-1 -- Databases

MS in Computational Analysis in Public Policy (Year 2) MS in Molecular Engineering MA in Computational Social Science (Year 2) Bx/MS in Computer Science (Option 2: Professionally-oriented - CS Majors) Bx/MS in Computer Science (Option 3: Profesionally-oriented - Non-CS Majors) Masters Program in Computer Science