Skip to content

CMU 10-414/714: Deep Learning Systems

Course Overview

  • University: Carnegie Mellon University (CMU)
  • Prerequisites: Introduction to Systems (e.g., 15-213), Basics of Deep Learning, Fundamental Mathematical Knowledge
  • Programming Languages: Python, C++
  • Difficulty: 🌟🌟🌟
  • Estimated Hours: 100 hours

The rise of deep learning owes much to user-friendly frameworks like PyTorch and TensorFlow. Yet, many users remain unfamiliar with these frameworks' internals. If you're curious or aspiring to delve into deep learning framework development, this course is an excellent starting point.

Covering the full spectrum of deep learning systems, the curriculum spans top-level framework design, autodifferentiation principles, hardware acceleration, and real-world deployment. The hands-on experience includes five assignments, building a deep learning library called Needle. Needle supports automatic differentiation, GPU acceleration, and various neural networks like CNNs, RNNs, LSTMs, and Transformers.

Even for beginners, the course gradually covers simple classification and backpropagation optimization. Detailed Jupyter notebooks accompany complex neural networks, providing insights. For those with foundational knowledge, assignments post autodifferentiation are approachable, offering new understandings.

Instructors Zico Kolter and Tianqi Chen released open-source content. Online evaluations and forums are closed, but local testing in framework code remains. Hope for an online version next fall.

Course Resources

Resource Compilation

All resources and assignment implementations used by @PKUFlyingPig in this course are consolidated in PKUFlyingPig/CMU10-714 - GitHub