CMU 11-711: Advanced Natural Language Processing (ANLP)
Course Overview
- University: Carnegie Mellon University
- Prerequisites: No strict prerequisites, but students should have experience with Python programming, as well as a background in probability and linear algebra. Prior experience with neural networks is recommended.
- Programming Language: Python
- Course Difficulty: 🌟🌟🌟🌟
- Estimated Workload: 100 hours
This is a graduate-level course covering both foundational and advanced topics in Natural Language Processing (NLP). The syllabus spans word representations, sequence modeling, attention mechanisms, Transformer architectures, and cutting-edge topics such as large language model pretraining, instruction tuning, complex reasoning, multimodality, and model safety. Compared to similar courses, this course stands out for the following reasons:
- Comprehensive and research-driven content: In addition to classical NLP methods, it offers in-depth discussions of recent trends and state-of-the-art techniques such as LLaMa and GPT-4.
- Strong practical component: Each lecture includes code demonstrations and online quizzes, and the final project requires reproducing and improving upon a recent research paper.
- Highly interactive: Active engagement is encouraged through Piazza discussions, Canvas quizzes, and in-class Q&A, resulting in an immersive and well-paced learning experience.
Self-study tips:
- Read the recommended papers before class and follow the reading sequence step-by-step.
- Set up a Python environment and become familiar with PyTorch and Hugging Face, as many hands-on examples are based on these frameworks.
Course Resources
- Course Website: https://www.phontron.com/class/anlp-fall2024/
- Course Videos: Lecture recordings are available on Canvas (CMU login required)
- Course Texts: Selected classical and cutting-edge research papers + chapters from A Primer on Neural Network Models for Natural Language Processing by Yoav Goldberg
- Course Assignments: https://www.phontron.com/class/anlp-fall2024/assignments/