# CMU 11-711: Advanced Natural Language Processing (ANLP)

## Course Overview

* University: Carnegie Mellon University  
* Prerequisites: No strict prerequisites, but students should have experience with Python programming, as well as a background in probability and linear algebra. Prior experience with neural networks is recommended.  
* Programming Language: Python  
* Course Difficulty: 🌟🌟🌟🌟  
* Estimated Workload: 100 hours  

This is a graduate-level course covering both foundational and advanced topics in Natural Language Processing (NLP). The syllabus spans word representations, sequence modeling, attention mechanisms, Transformer architectures, and cutting-edge topics such as large language model pretraining, instruction tuning, complex reasoning, multimodality, and model safety. Compared to similar courses, this course stands out for the following reasons:

1. **Comprehensive and research-driven content**: In addition to classical NLP methods, it offers in-depth discussions of recent trends and state-of-the-art techniques such as LLaMa and GPT-4.  
2. **Strong practical component**: Each lecture includes code demonstrations and online quizzes, and the final project requires reproducing and improving upon a recent research paper.  
3. **Highly interactive**: Active engagement is encouraged through Piazza discussions, Canvas quizzes, and in-class Q&A, resulting in an immersive and well-paced learning experience.

Self-study tips:

* Read the recommended papers before class and follow the reading sequence step-by-step.  
* Set up a Python environment and become familiar with PyTorch and Hugging Face, as many hands-on examples are based on these frameworks.

## Course Resources

* Course Website: [https://www.phontron.com/class/anlp-fall2024/](https://www.phontron.com/class/anlp-fall2024/)  
* Course Videos: Lecture recordings are available on Canvas (CMU login required)  
* Course Texts: Selected classical and cutting-edge research papers + chapters from *A Primer on Neural Network Models for Natural Language Processing* by Yoav Goldberg  
* Course Assignments: [https://www.phontron.com/class/anlp-fall2024/assignments/](https://www.phontron.com/class/anlp-fall2024/assignments/)
