mirror of
https://github.com/PKUFlyingPig/cs-self-learning.git
synced 2026-06-21 00:47:16 +08:00
Compare commits
27 commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3ec3100168 | ||
|
|
ed6ce78047 | ||
|
|
cb90db338b | ||
|
|
f45cd57844 | ||
|
|
4739f68318 | ||
|
|
9519b3a667 | ||
|
|
fbf8f26a2b | ||
|
|
27586ff904 | ||
|
|
ac0cef182a | ||
|
|
542a7f4a9b | ||
|
|
188a8f88c3 | ||
|
|
5956926395 | ||
|
|
ddc06bd6e5 | ||
|
|
c116aae45e | ||
|
|
cf393ddff2 | ||
|
|
8c9db49c96 | ||
|
|
9f68cb64d0 | ||
|
|
3bb5a900e7 | ||
|
|
25e0b34359 | ||
|
|
c9365b2ed9 | ||
|
|
6162cd1b9c | ||
|
|
0e793039d8 | ||
|
|
5d57ab9b4e | ||
|
|
816342af1b | ||
|
|
43a37646ab | ||
|
|
95fc541954 | ||
|
|
3432316a0d |
46 changed files with 569 additions and 72 deletions
|
|
@ -4,7 +4,7 @@
|
|||
|
||||
# CS 自学指南
|
||||
|
||||
> *Everyone should enjoy CS if you have a good teacher to teach you a good course.*
|
||||
> _Everyone should enjoy CS if you have a good teacher to teach you a good course._
|
||||
|
||||
<a href="https://trendshift.io/repositories/4643" target="_blank"><img src="https://trendshift.io/api/badge/repositories/4643" alt="PKUFlyingPig%2Fcs-self-learning | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
|
||||
|
|
@ -13,7 +13,6 @@
|
|||
[](https://github.com/PKUFlyingPig/cs-self-learning/issues)
|
||||
[](https://github.com/PKUFlyingPig/cs-self-learning)
|
||||
|
||||
|
||||
新冠肆虐,网课当道,CS 爆火,这一系列的事件都让自学计算机成为了一种潮流。
|
||||
|
||||
随着欧美众多名校将质量极高的计算机课程全部开源,自学 CS 成了一件可操作性极强的事情。毫不夸张地说,只要你有毅力和兴趣,自学的成果完全不亚于你在国内任何一所大学受到的本科 CS 教育(当然,这里单指计算机专业领域,大学带给你的显然不止是专业知识)。
|
||||
|
|
@ -22,7 +21,7 @@
|
|||
|
||||
但同时,自学这条路也有很多困难和阻力:课程繁多不知如何选择,资料零散甚至残缺,作业难度不知深浅,课内任务还需要花时间应付······这些主客观因素叠加到一起,使得好课虽多,却只能在收藏夹里吃灰。
|
||||
|
||||
在大学的第四个年头,我想把这一路自学走来的经验和教训,把那些让我受益终身的课程记录下来,分享给大家,形成了这本CS自学指南,以期能给所有想自学计算机的朋友一点帮助。
|
||||
在大学的第四个年头,我想把这一路自学走来的经验和教训,把那些让我受益终身的课程记录下来,分享给大家,形成了这本 CS 自学指南,以期能给所有想自学计算机的朋友一点帮助。
|
||||
|
||||
我的目标是让一个刚刚接触计算机的小白,可以完全凭借这些开源社区的优质资源,少走弯路,在 2-3 年内成长为一个有扎实的数学功底和代码能力,经历过数十个千行代码量的 Project 的洗礼,掌握至少 C/C++/Java/JS/Python/Go/Rust 等主流语言,对算法、电路、体系、网络、操统、编译、人工智能、机器学习、计算机视觉、自然语言处理、强化学习、密码学、信息论、博弈论、数值分析、统计学、分布式、数据库、图形学、Web 开发、云服务、超算等等方面均有所涉猎的全能程序员。此后,无论是选择科研还是就业,我相信你都会有相当的竞争力。
|
||||
|
||||
|
|
@ -30,7 +29,7 @@
|
|||
|
||||
## 如何成为贡献者
|
||||
|
||||
一个人的力量终究是有限的,对于书中任意章节你若有想要补充的内容,欢迎各位提出 [Pull Request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork)。如果你想贡献一门新的课程,可以参考目前 repo 中的 [template](./template.md) 文件作为模版,并在 [mkdocs.yml](./mkdocs.yml) 文件中添加其navigation,当然你还可以在 [CS 学习规划](./docs/CS学习规划.md) 里的对应模块为其添加言简意赅的导语。如果你有想推荐的书籍,请参考 [好书推荐](https://raw.githubusercontent.com/PKUFlyingPig/cs-self-learning/master/docs/%E5%A5%BD%E4%B9%A6%E6%8E%A8%E8%8D%90.md) 模块上方的注释按相应格式添加内容。
|
||||
一个人的力量终究是有限的,对于书中任意章节你若有想要补充的内容,欢迎各位提出 [Pull Request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork)。如果你想贡献一门新的课程,可以参考目前 repo 中的 [template](./template.md) 文件作为模版,并在 [mkdocs.yml](./mkdocs.yml) 文件中添加其 navigation,当然你还可以在 [CS 学习规划](./docs/CS学习规划.md) 里的对应模块为其添加言简意赅的导语。如果你有想推荐的书籍,请参考 [好书推荐](https://raw.githubusercontent.com/PKUFlyingPig/cs-self-learning/master/docs/%E5%A5%BD%E4%B9%A6%E6%8E%A8%E8%8D%90.md) 模块上方的注释按相应格式添加内容。
|
||||
|
||||
对于中英混合排版的要点规范,可以参考[这个仓库](https://github.com/sparanoid/chinese-copywriting-guidelines/blob/master/README.zh-Hans.md),我们将会对您的 Pull Request 做相应的校对,具体原因参见这个 [issue](https://github.com/PKUFlyingPig/cs-self-learning/issues/114)。
|
||||
|
||||
|
|
@ -49,6 +48,7 @@
|
|||
## ✨ 鸣谢
|
||||
|
||||
特别感谢 @[AlfredThiel](https://github.com/AlfredThiel) 为项目制作了精美的 Logo。
|
||||
|
||||
<!-- support by https://contrib.rocks -->
|
||||
<a href="https://github.com/PKUFlyingPig/cs-self-learning/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=PKUFlyingPig/cs-self-learning"/>
|
||||
|
|
|
|||
|
|
@ -330,6 +330,14 @@ The popularity of AlphaGo a few years ago brought deep learning to the public ey
|
|||
|
||||
Due to the rapid development of deep learning, there are now many research branches. For further in-depth study, consider the following representative courses:
|
||||
|
||||
If you want rigorous fundamentals, start with CMU 11-785: it is dense, practical, and has very little filler content. MIT 6.7960 provides broader coverage beyond mainstream LLM topics, including CV-oriented material, and its assignments/projects are feasible for self-learners. NYU DLSP21 is especially notable because it is taught by Yann LeCun, offering a rare public opportunity to follow a full deep learning course from him.
|
||||
|
||||
#### Fundamentals and Breadth
|
||||
|
||||
- [CMU 11-785: Introduction to Deep Learning](深度学习/CMU11-785.md)
|
||||
- [MIT 6.7960: Deep Learning](深度学习/MIT6-7960.md)
|
||||
- [NYU DLSP21: NYU Deep Learning Spring 2021](深度学习/NYU-DLSP21.md)
|
||||
|
||||
#### Computer Vision
|
||||
|
||||
- [UMich EECS 498-007 / 598-005: Deep Learning for Computer Vision](深度学习/EECS498-007.md)
|
||||
|
|
|
|||
|
|
@ -323,6 +323,16 @@ Berkeley 作为著名开源数据库 postgres 的发源地也不遑多让,[UCB
|
|||
|
||||
当然因为深度学习领域发展非常迅速,已经拥有了众多研究分支,如果想要进一步深入,可以按需学习下面罗列的代表课程:
|
||||
|
||||
如果你想系统打牢基础,可以优先看 CMU 11-785:这门课风格非常扎实,内容一点不水,作业也能很好地锻炼训练与调参能力;MIT 6.7960 的覆盖面更广,除了主流 LLM 路线之外也兼顾 CV 等方向,作业和项目对自学者也比较友好;NYU DLSP21 的亮点则是 Yann LeCun 亲自授课,对很多人来说这是非常难得的公开系统课程资源。
|
||||
|
||||
#### 综合与基础
|
||||
|
||||
[CMU 11-785: Introduction to Deep Learning](深度学习/CMU11-785.md)
|
||||
|
||||
[MIT 6.7960: Deep Learning](深度学习/MIT6-7960.md)
|
||||
|
||||
[NYU DLSP21: NYU Deep Learning Spring 2021](深度学习/NYU-DLSP21.md)
|
||||
|
||||
#### 计算机视觉
|
||||
|
||||
[UMich EECS 498-007 / 598-005: Deep Learning for Computer Vision](深度学习/EECS498-007.md)
|
||||
|
|
|
|||
|
|
@ -21,4 +21,4 @@ Karpathy holds a **PhD from Stanford University**, where he worked on convolutio
|
|||
- **Lecture Videos:** [YouTube Playlist](https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ)
|
||||
- **Assignments:** Self-guided projects and code implementation exercises available throughout the lectures
|
||||
|
||||
For more information, watch the full playlist on YouTube.
|
||||
For more information, watch the full playlist on YouTube.
|
||||
|
|
@ -21,4 +21,4 @@ Karpathy 拥有 **斯坦福大学博士学位**,师从 **Fei-Fei Li(李飞
|
|||
- **课程视频:** [YouTube 播放列表](https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ)
|
||||
- **作业:** 课程中提供的代码实践和项目练习
|
||||
|
||||
更多信息请访问 YouTube 观看完整课程视频。
|
||||
更多信息请访问 YouTube 观看完整课程视频。
|
||||
|
|
@ -14,8 +14,8 @@ The goal of this course is to provide a deep understanding of the fundamental pr
|
|||
|
||||
## Resources
|
||||
|
||||
- Course Website: [CMU15418](http://15418.courses.cs.cmu.edu/spring2016/), [CS149](https://gfxcourses.stanford.edu/cs149/fall21)
|
||||
- Recordings: [CMU15418](http://15418.courses.cs.cmu.edu/spring2016/lectures), [CS149](https://youtube.com/playlist?list=PLoROMvodv4rMp7MTFr4hQsDEcX7Bx6Odp&si=txtQiRDZ9ZZUzyRn)
|
||||
- Course Website: [CMU15418](https://www.cs.cmu.edu/afs/cs/academic/class/15418-s18/www/index.html), [CS149](https://gfxcourses.stanford.edu/cs149/fall21)
|
||||
- Recordings: [CMU15418](https://www.cs.cmu.edu/afs/cs/academic/class/15418-s18/www/schedule.html), [CS149](https://youtube.com/playlist?list=PLoROMvodv4rMp7MTFr4hQsDEcX7Bx6Odp&si=txtQiRDZ9ZZUzyRn)
|
||||
- Textbook: None
|
||||
- Assignments: <https://gfxcourses.stanford.edu/cs149/fall21>, 5 assignments.
|
||||
|
||||
|
|
|
|||
|
|
@ -14,8 +14,8 @@
|
|||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:[CMU15418](http://15418.courses.cs.cmu.edu/spring2016/), [CS149](https://gfxcourses.stanford.edu/cs149/fall21)
|
||||
- 课程视频:[CMU15418](http://15418.courses.cs.cmu.edu/spring2016/lectures), [CS149](https://youtube.com/playlist?list=PLoROMvodv4rMp7MTFr4hQsDEcX7Bx6Odp&si=txtQiRDZ9ZZUzyRn)
|
||||
- 课程网站:[CMU15418](https://www.cs.cmu.edu/afs/cs/academic/class/15418-s18/www/index.html), [CS149](https://gfxcourses.stanford.edu/cs149/fall21)
|
||||
- 课程视频:[CMU15418](https://www.cs.cmu.edu/afs/cs/academic/class/15418-s18/www/schedule.html), [CS149](https://youtube.com/playlist?list=PLoROMvodv4rMp7MTFr4hQsDEcX7Bx6Odp&si=txtQiRDZ9ZZUzyRn)
|
||||
- 课程教材:无
|
||||
- 课程作业:<https://gfxcourses.stanford.edu/cs149/fall21>,5 个编程作业
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,8 @@ Git is a powerful tool and when you finally master it, you will find all the eff
|
|||
Different from Vim, I don't suggest beginners use Git rashly without fully understanding it, because its inner logic can not be acquainted by practicing. Here is my recommended learning path:
|
||||
|
||||
1. Read this [Git tutorial](https://missing.csail.mit.edu/2020/version-control/) in English, or you can watch this [Git tutorial (by 尚硅谷)](https://www.bilibili.com/video/BV1vy4y1s7k6) in Chinese.
|
||||
2. Read Chap1 - Chap5 of this open source book [Pro Git](https://git-scm.com/book/en/v2). Yes, to learn Git, you need to read a book.
|
||||
3. Now that you have understood its principles and most of its usages, it's time to consolidate those commands by practicing. How to use Git properly is a kind of philosophy. I recommend reading this blog [How to Write a Git Commit Message](https://chris.beams.io/posts/git-commit/).
|
||||
4. You are now in love with Git and are not content with only using it, you want to build a Git by yourself! Great, that's exactly what I was thinking. [This tutorial](https://wyag.thb.lt/) will satisfy you!
|
||||
5. What? Building your own Git is not enough? Seems that you are also passionate about reinventing the wheels. These two GitHub projects, [build-your-own-x](https://github.com/danistefanovic/build-your-own-x) and [project-based-learning](https://github.com/tuvtran/project-based-learning), collected many wheel-reinventing tutorials, e.g., text editor, virtual machine, docker, TCP and so on.
|
||||
2. Read Chap1 - Chap5 of this open source book [Pro Git](https://git-scm.com/book/en/v2). Yes, to learn Git, you need to read a book.
|
||||
3. [Learn Git Branching](https://learngitbranching.js.org/) is an interactive Git learning website that can help you quickly get started with using Git.
|
||||
4. Now that you have understood its principles and most of its usages, it's time to consolidate those commands by practicing. How to use Git properly is a kind of philosophy. I recommend reading this blog [How to Write a Git Commit Message](https://chris.beams.io/posts/git-commit/).
|
||||
5. You are now in love with Git and are not content with only using it, you want to build a Git by yourself! Great, that's exactly what I was thinking. [This tutorial](https://wyag.thb.lt/) will satisfy you!
|
||||
6. What? Building your own Git is not enough? Seems that you are also passionate about reinventing the wheels. These two GitHub projects, [build-your-own-x](https://github.com/danistefanovic/build-your-own-x) and [project-based-learning](https://github.com/tuvtran/project-based-learning), collected many wheel-reinventing tutorials, e.g., text editor, virtual machine, docker, TCP and so on.
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ Git 的设计非常优雅,但初学者通常因为很难理解其内部逻辑
|
|||
|
||||
1. 阅读这篇 [Git tutorial](https://missing.csail.mit.edu/2020/version-control/),视频的话可以看这个[尚硅谷Git教程](https://www.bilibili.com/video/BV1vy4y1s7k6)
|
||||
2. 阅读这本开源书籍 [Pro Git](https://git-scm.com/book/en/v2) 的 Chapter1 - Chapter5,是的没错,学 Git 需要读一本书(捂脸)。
|
||||
3. 此时你已经掌握了 Git 的原理和绝大部分用法,接下来就可以在实践中反复巩固 Git 的命令了。但用好它同样是一门哲学,我个人觉得这篇[如何写好 Commit Message](https://chris.beams.io/posts/git-commit/) 的博客非常值得一读。
|
||||
4. 好的此时你已经爱上了 Git,你已经不满足于学会它了,你想自己实现一个 Git!巧了,我当年也有这样的想法,[这篇 tutorial](https://wyag.thb.lt/) 可以满足你!
|
||||
5. 什么?光实现一个 Git 无法满足你?小伙子/小仙女有前途,巧的是我也喜欢造轮子,这两个 GitHub 项目 [build-your-own-x](https://github.com/danistefanovic/build-your-own-x) 和 [project-based-learning](https://github.com/tuvtran/project-based-learning) 收录了你能想到的各种造轮子教程,比如:自己造个编辑器、自己写个虚拟机、自己写个 docker、自己写个 TCP 等等等等。
|
||||
3. [Learn Git Branching](https://learngitbranching.js.org/) 是一个交互式的 Git 学习网站, 可以帮助你快速上手 Git 的使用。
|
||||
4. 此时你已经掌握了 Git 的原理和绝大部分用法,接下来就可以在实践中反复巩固 Git 的命令了。但用好它同样是一门哲学,我个人觉得这篇[如何写好 Commit Message](https://chris.beams.io/posts/git-commit/) 的博客非常值得一读。
|
||||
5. 好的此时你已经爱上了 Git,你已经不满足于学会它了,你想自己实现一个 Git!巧了,我当年也有这样的想法,[这篇 tutorial](https://wyag.thb.lt/) 可以满足你!
|
||||
6. 什么?光实现一个 Git 无法满足你?小伙子/小仙女有前途,巧的是我也喜欢造轮子,这两个 GitHub 项目 [build-your-own-x](https://github.com/danistefanovic/build-your-own-x) 和 [project-based-learning](https://github.com/tuvtran/project-based-learning) 收录了你能想到的各种造轮子教程,比如:自己造个编辑器、自己写个虚拟机、自己写个 docker、自己写个 TCP 等等等等。
|
||||
|
|
|
|||
|
|
@ -71,11 +71,13 @@
|
|||
- [Python3 Documentation](https://docs.python.org/zh-cn/3/): Official Chinese documentation for Python3.
|
||||
- [C++ Reference](https://en.cppreference.com/w/): C++ reference manual.
|
||||
- [OI Wiki](https://oi-wiki.org/): An integrated site for programming competition knowledge.
|
||||
- [CTF Wiki](https://ctf-wiki.org/): An integrated site for knowledge and tools related to cybersecurity competitions.
|
||||
- [Microsoft Learn](https://learn.microsoft.com/zh-cn/): Microsoft's official learning platform, containing most Microsoft product documentation.
|
||||
- [Arch Wiki](https://wiki.archlinux.org/): Wiki written for Arch Linux, containing a lot of Linux-related knowledge.
|
||||
- [Qt Wiki](https://wiki.qt.io/Main): Official Qt Wiki.
|
||||
- [OpenCV Chinese Documentation](https://opencv.apachecn.org/#/): Community version of OpenCV's Chinese documentation.
|
||||
- [npm Docs](https://docs.npmjs.com/): Official npm documentation.
|
||||
- [developer-roadmap](https://roadmap.sh/): provides roadmaps, guides and other educational content to help guide developers in picking up a path and guide their learnings.
|
||||
|
||||
## Communication Platforms
|
||||
|
||||
|
|
|
|||
|
|
@ -71,11 +71,13 @@
|
|||
- [Python3 Documentation](https://docs.python.org/zh-cn/3/): Python3 官方中文文档。
|
||||
- [C++ Reference](https://en.cppreference.com/w/): C++ 参考手册。
|
||||
- [OI Wiki](https://oi-wiki.org/): 编程竞赛知识整合站点。
|
||||
- [CTF Wiki](https://ctf-wiki.org/):网络安全竞赛相关知识与工具的整合站点。
|
||||
- [Microsoft Learn](https://learn.microsoft.com/zh-cn/): 微软官方的学习平台,包含了绝大多数微软产品的文档。
|
||||
- [Arch Wiki](https://wiki.archlinux.org/): 专为 Arch Linux 而写的 Wiki,包含了大量 Linux 相关的知识。
|
||||
- [Qt Wiki](https://wiki.qt.io/Main): Qt 官方 Wiki。
|
||||
- [OpenCV 中文文档](https://opencv.apachecn.org/#/): OpenCV 的社区版中文文档。
|
||||
- [npm Docs](https://docs.npmjs.com/): npm 官方文档。
|
||||
- [developer-roadmap](https://roadmap.sh/):帮助开发者了解学习路径并在职业生涯中不断成长。
|
||||
|
||||
## 交流平台
|
||||
- [GitHub](https://github.com/): 许多开源项目的托管平台,也是许多开源项目的主要交流平台,通过查看 issue 可以解决许多问题。
|
||||
|
|
|
|||
|
|
@ -3,30 +3,49 @@
|
|||
## Descriptions
|
||||
|
||||
- Offered by: UC Berkeley
|
||||
- Prerequisites: CS61A, CS61B, CS61C
|
||||
- Prerequisites: CS61A, CS61B, CS61C; Solid C programming and GDB debugging skills
|
||||
- Programming Languages: C, x86 Assembly
|
||||
- Difficulty: 🌟🌟🌟🌟🌟🌟
|
||||
- Class Hour: 200 hours+
|
||||
|
||||
The course impressed me in two aspects:
|
||||
|
||||
Firstly, the textbook: *Operating Systems: Principles and Practice (2nd Edition)* is written in an insightful but easy-to-understand way, well compensated for the lack of theoretical knowledge in MIT6.S081, I highly recommend you to read this book.
|
||||
Firstly, the textbook: Operating Systems: Principles and Practice (2nd Edition) consists of four volumes. It is written in a very accessible yet profound way, with vivid and sometimes even humorous language. It serves as an excellent supplement to the lecture videos and perfectly fills some theoretical gaps left by MIT 6.S081. This is also the experimental textbook for CMU's OS course (15410), I highly recommend reading it! Related resources are shared in the "Classic Books" section of this repository.
|
||||
|
||||
Secondly, the project for this course *Pintos* is a great journey for system hackers. *Pintos* is a toy operating system developed at Stanford for educational use. The author Ben Pfaff even published a [paper](https://benpfaff.org/papers/pintos.pdf) to explain the design principles of *Pintos*.
|
||||
|
||||
Unlike the small but comprehensive design philosophy in MIT's xv6 labs, *Pintos* emphasizes system design and implementation more. The codebase is about 10,000 LOC and only provides the basic functions of a working operating system. The four projects let you add scheduler (Project1), system calls (Project2), virtual memory (Project3), and the file system (Project4) to this extremely simple operating system. All projects leave a a big design space for students and require more than 2000 LOC. Based on the [feedback](https://www.quora.com/What-is-it-like-to-take-CS-140-Operating-Systems-at-Stanford) from Stanford students, the latter two projects take over 40 hours per person even in teams of 3-4 people.
|
||||
Unlike the small but comprehensive design philosophy in MIT's xv6 labs, *Pintos* emphasizes system design and implementation more. The codebase is about 10,000 LOC and only provides the basic functions of a working operating system. Each project has almost no boilerplate code; you must design the implementation yourself and weigh the pros and cons of different schemes. The four projects let you add scheduler (Project1), system calls (Project2), virtual memory (Project3), and the file system (Project4) *(Note: The specific requirements for CS162 Pintos differ slightly; see the assignment description below)* to this extremely simple operating system. All projects leave a a big design space for students and require more than 2000 LOC. Based on the [feedback](https://www.quora.com/What-is-it-like-to-take-CS-140-Operating-Systems-at-Stanford) from Stanford students, the latter two projects take over 40 hours per person even in teams of 3-4 people.
|
||||
|
||||
Although it is tough, Stanford, Berkeley, JHU and many other top U.S. colleges have chosen *Pintos* as their OS course project. If you're really interested in operating systems, it will greatly improve your ability to write and debug low-level system code. For me, it is an invaluable experience to design, implement, and debug a large system independently.
|
||||
Although it is tough, Stanford, Berkeley, JHU and many other top U.S. colleges have chosen *Pintos* as their OS course project. If you're really interested in operating systems, it will greatly improve your ability to write and debug low-level system code and teach you how to design a system by making trade-offs between different possibilities. For me, it is an invaluable experience to design, implement, and debug a large system independently.
|
||||
|
||||
*Pintos* will also be introduced as a course project in Peking University's OS Course. In the Spring 2022 semester, I worked with [another TA](https://github.com/AlfredThiel) to write a comprehensive [lab documentation](https://pkuflyingpig.gitbook.io/pintos) and provided a docker image for the ease of cross-platform development. In the last semester before graduation, I hope such an attempt can make more people fall in love with systems and contribute to the field of systems in China.
|
||||
|
||||
## Course Resources
|
||||
|
||||
- Course Website: <https://cs162.org/>
|
||||
- Lecture Videos: <https://www.youtube.com/watch?v=YfHY0pvpRkk>, videos for each lecture can be found on the course website.
|
||||
- Textbook: [Operating Systems: Principles and Practice (2nd Edition)](http://ospp.cs.washington.edu/)
|
||||
- Assignments: <https://cs162.org/>, 6 Homework, 3 Projects, the course website has specific requirements.
|
||||
- Course Website:
|
||||
- [Current Semester](https://cs162.org/)
|
||||
- [Fa25 - Wayback Machine](https://web.archive.org/web/20251211080516/https://cs162.org/)
|
||||
- Lecture Videos: Currently, three semesters are publicly available: Fall 2020, Fall 2021, and Spring 2022. Based on my experience with the Fall 2025 semester, the **Spring 2022** version is best for self-study. It was recorded in-person (except for the first four lectures), featuring more student-teacher interaction and many valuable questions addressed in class:
|
||||
- [Spring 2022 Lecture Videos (Bilibili)](https://www.bilibili.com/video/BV1L541117gr?vd_source=e293470ea109e008c4d9516e39ef318f&p=5&spm_id_from=333.788.videopod.episodes)
|
||||
- [Fall 2020 Lecture Videos (Bilibili)](https://www.bilibili.com/video/BV1MwDSYWEKy?spm_id_from=333.788.videopod.sections&vd_source=e293470ea109e008c4d9516e39ef318f&p=24)
|
||||
- Fall 2021 video links can be found on the [Fall 2021 Website](https://web.archive.org/web/20211216005317/https://cs162.org/).
|
||||
- Textbook: [Operating Systems: Principles and Practice (2nd Edition)](http://ospp.cs.washington.edu/). This textbook is an excellent supplement to the lectures and is highly recommended.
|
||||
- Assignments: Consists of 3 Projects and 6 Homeworks. (The workload for each Homework is roughly equivalent to a full Project in most other open-source courses. Projects were originally designed for teams; self-studying them alone involves a significant workload):
|
||||
- 3 Projects (each with complete local tests):
|
||||
1. User Programs: Implement argument parsing for process execution, process-related system calls (including the `fork` syscall added in 2025), and file-related system calls.
|
||||
2. Threads: Implement a non-busy-waiting `timer_sleep` function, a strict priority scheduler, multi-threading support, and a simplified `pthread` library. (Note: This differs from the Multi-Level Feedback Queue requirements in the PKU Pintos and Stanford CS212 versions).
|
||||
3. File Systems: Implement a kernel Buffer Cache, extensible files, and subdirectories.
|
||||
- 6 Homeworks: Includes a sub-task for the MapReduce assignment. Both HTTP and MapReduce assignments have two versions: C and Rust. Except for the Memory lab, most Homeworks lack local autograders (though most can be manually tested effectively, except for MapReduce, which can be replaced by the [MIT 6.824 MapReduce lab](https://pdos.csail.mit.edu/6.824/labs/lab-mr.html)).
|
||||
1. List: Familiarize with the built-in linked list structure in Pintos.
|
||||
2. Shell: Implement a Shell supporting directory commands, program execution, path parsing, redirection, pipes, and signal handling.
|
||||
3. HTTP: Implement an HTTP server supporting GET requests.
|
||||
4. Memory: Implement memory management functions like `sbrk` and `malloc`.
|
||||
5. MapReduce: Implement a fault-tolerant MapReduce system (including the RPC Lab).
|
||||
|
||||
---
|
||||
|
||||
## Personal Resources
|
||||
|
||||
All resources and assignment implementations (including code, design documents, and starter code) used during my study of the **Fall 2025** semester are summarized in the [@RisingUppercut/CS162-fall25 - GitHub](https://github.com/RisingUppercut/UCB_CS162_2025Fall) repository.
|
||||
|
||||
Since the Operating System Course at PKU uses the project, my implementation is not open source to prevent plagiarism.
|
||||
|
|
|
|||
|
|
@ -3,32 +3,53 @@
|
|||
## 课程简介
|
||||
|
||||
- 所属大学:UC Berkeley
|
||||
- 先修要求:CS61A, CS61B, CS61C
|
||||
- 先修要求:CS61A, CS61B, CS61C,扎实的C语言能力及GDB调试能力
|
||||
- 编程语言:C, x86汇编
|
||||
- 课程难度:🌟🌟🌟🌟🌟🌟
|
||||
- 预计学时:200 小时+,上不封顶
|
||||
|
||||
这门课让我记忆犹新的有两个部分:
|
||||
|
||||
首先是教材,这本书用的教材 *Operating Systems: Principles and Practice (2nd Edition)* 一共四卷,写得非常深入浅出,很好地弥补了 MIT6.S081 在理论知识上的些许空白,非常建议大家阅读。相关资源会分享在本书的经典书籍推荐模块。
|
||||
首先是教材,这本书用的教材 *Operating Systems: Principles and Practice (2nd Edition)* 一共四卷,写得非常深入浅出,语言生动甚至时而幽默,是本课程 Lecture 视频内容极好的完善与补充,同时也很好地弥补了 MIT6.S081 在理论知识上的些许空白,是 CMU 操作系统课 15410 的实验性教材,非常建议大家阅读!相关资源会分享在本书的经典书籍推荐模块。
|
||||
|
||||
其次是这门课的 Project —— Pintos。Pintos 是由 Ben Pfaff 等人在 x86 平台上编写的教学用操作系统,Ben Pfaff 甚至专门发了篇 [paper](https://benpfaff.org/papers/pintos.pdf) 来阐述 Pintos 的设计思想。
|
||||
|
||||
和 MIT 的 xv6 小而精的 lab 设计理念不同,Pintos 更注重系统的 Design and Implementation。Pintos 本身仅一万行左右,只提供了操作系统最基本的功能。而 4 个Project,就是让你在这个极为精简的操作系统之上,分别为其增加线程调度机制 (Project1),系统调用 (Project2),虚拟内存 (Project3) 以及文件系统 (Project4)。所有的 Project 都给学生留有很大的设计空间,总代码量在 2000 行左右。根据 Stanford 学生[自己的反馈][quora_link],在 3-4 人组队的情况下,后两个 Project 的人均耗时也在 40 个小时以上。
|
||||
和 MIT 的 xv6 小而精的 lab 设计理念不同,Pintos 更注重系统的 Design and Implementation。Pintos 本身仅一万行左右,只提供了操作系统最基本的功能。每个project几乎没有框架代码,都需要自己设计实现并权衡不同方案的优缺点。而 4 个Project,就是让你在这个极为精简的操作系统之上,分别为其增加线程调度机制 (Project1),系统调用 (Project2),虚拟内存 (Project3) 以及文件系统 (Project4)*(注:CS162 Pintos 的 project 和上述略有不同,详见下方课程作业说明)*。所有的 Project 都给学生留有很大的设计空间,总代码量在 2000 行左右。根据 Stanford 学生[自己的反馈][quora_link],在 3-4 人组队的情况下,后两个 Project 的人均耗时也在 40 个小时以上。
|
||||
|
||||
[quora_link]: https://www.quora.com/What-is-it-like-to-take-CS-140-Operating-Systems-at-Stanford
|
||||
|
||||
虽然难度很大,但 Stanford, Berkeley, JHU 等多所美国顶尖名校的操统课程均采用了 Pintos。如果你真的对操作系统很感兴趣,Pintos 会极大地提高你编写和 debug 底层系统代码的能力。在本科阶段,能自己设计、实现并 debug 一个大型系统,是一段非常珍贵的经历。
|
||||
虽然难度很大,但 Stanford, Berkeley, JHU 等多所美国顶尖名校的操统课程均采用了 Pintos。如果你真的对操作系统很感兴趣,Pintos 会极大地提高你编写和 debug 底层系统代码的能力,并让你学会设计一个系统,使你在不同可能的设计中学会取舍。在本科阶段,能自己设计、实现并 debug 一个大型系统,是一段非常珍贵的经历。
|
||||
|
||||
北大 2022 年春季学期的操作系统实验班也将会首次引入 Pintos 作为课程 Project。我和该课程的[另一位助教](https://github.com/AlfredThiel)整理并完善了 Pintos 的[实验文档](https://pkuflyingpig.gitbook.io/pintos),并利用 Docker 配置了跨平台的实验环境,想自学的同学可以按文档自行学习。在毕业前的最后一个学期,希望能用这样的尝试,让更多人爱上系统领域,为国内的系统研究添砖加瓦。
|
||||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:<https://cs162.org/>
|
||||
- 课程视频:<https://www.youtube.com/watch?v=YfHY0pvpRkk>,每节课的链接参见课程网站
|
||||
- 课程教材:[Operating Systems: Principles and Practice (2nd Edition)](http://ospp.cs.washington.edu/)
|
||||
- 课程作业:<https://cs162.org/>,6 个 Homework, 3 个 Project,具体要求参见课程网站
|
||||
- 课程网站:
|
||||
- [当前最新学期](https://cs162.org/)
|
||||
- [Fa25-WayBack Machine](https://web.archive.org/web/20251211080516/https://cs162.org/)
|
||||
- 课程视频,目前公开的视频有三个学期,分别为: 2020Fall,2021Fall 及 2022Spring 。根据我学习 2025Fall 学期的经历来看,2022Spring 的最适合自学,因为这个学期是线下录制的形式(除了前四节),上课时师生之间的互动更多,有很多有价值的问题在课堂上被解决:
|
||||
- [2022Spring课程视频](https://www.bilibili.com/video/BV1L541117gr?vd_source=e293470ea109e008c4d9516e39ef318f&p=5&spm_id_from=333.788.videopod.episodes)
|
||||
- [2020Fall课程视频](https://www.bilibili.com/video/BV1MwDSYWEKy?spm_id_from=333.788.videopod.sections&vd_source=e293470ea109e008c4d9516e39ef318f&p=24)
|
||||
- 2021Fall的各个视频链接在[2021Fall网站](https://web.archive.org/web/20211216005317/https://cs162.org/)上
|
||||
- 课程教材:[Operating Systems: Principles and Practice (2nd Edition)](http://ospp.cs.washington.edu/),本教材是课上 Lecture 内容的很好的补充,强烈推荐阅读。
|
||||
- 课程作业:3 个 Project,6 个 Homework(每个Homework的工作量大致相当于其他大部分公开课的Project, Project原本要求是组队实现,一个人自学的工作量较大):
|
||||
- 3 个 Project , 每个 Project 都有完整的本地测试:
|
||||
1. User Programs: 实现进程执行函数的参数解析传递,实现进程相关的系统调用(25年的新增了fork系统调用),实现文件相关系统调用。
|
||||
2. Threads: 实现不忙等的 `timer_sleep` 函数, 实现严格优先级调度器,实现对多线程的支持,实现简化版的 pthread 库(这与北大的 Pintos 及 斯坦福的 CS212 的多级反馈调度的要求不同)。
|
||||
3. File Systems: 实现文件系统内核缓冲区 Buffer Cache,实现可扩容的文件,实现子目录。
|
||||
- 6 个 Homework, 其中一个为 Map Reduce 作业的子任务,作业 HTTP 及 Map Reduce 均有两个版本:C 和 Rust。除了 Memory 作业外,其他 Homework 均没有本地测试(但除了 Map Reduce 作业外, 其他作业都可以手动测试的大差不差, Map Reduce 作业可换成 [MIT 6.824 的对应作业](https://pdos.csail.mit.edu/6.824/labs/lab-mr.html))
|
||||
1. List: 熟悉 Pintos 内置的链表结构
|
||||
2. Shell: 实现支持目录命令、启动程序、路径解析、重定向、管道、信号处理的 Shell
|
||||
3. HTTP: 实现一个支持 HTTP GET 请求的 HTTP 服务器
|
||||
4. Memory: 实现 sbrk,malloc 等内存管理函数
|
||||
5. Map Reduce: 实现一个可容忍错误的 MapReduce 系统
|
||||
- 包含作业 RPC Lab
|
||||
|
||||
|
||||
## 资源汇总
|
||||
|
||||
@[RisingUppercut] 在学习这门课(2025Fall)中用到的所有资源和作业实现(包括代码、设计文档、初始框架代码等)都汇总在 [@RisingUppercut/CS162-fall25 - GitHub](https://github.com/RisingUppercut/UCB_CS162_2025Fall) 中。
|
||||
|
||||
[RisingUppercut]: https://github.com/RisingUppercut
|
||||
|
||||
由于北大的操统实验班采用了该课程的 Project,为了防止代码抄袭,我的代码实现没有开源。
|
||||
|
|
|
|||
|
|
@ -12,6 +12,8 @@ It is the second course of UC Berkeley's CS61 series. It mainly focuses on the d
|
|||
|
||||
I took the version for 2018 Spring. Josh Hug, the instructor, generously made the autograder open-source. You can use [gradescope](https://gradescope.com/) invitation code published on the website for free and easily test your implementation.
|
||||
|
||||
According to the professor's latest policy, SP2021 CS61B is now open to the public. To get everything set up, go to Gradescope and select the "Add a course" button. Enter course code **MB7ZPY** to be added.
|
||||
|
||||
All programming assignments in this course are done in Java. Students without Java experience don't have to worry. There will be detailed tutorials in the course from the configuration of IDEA to the core syntax and features of Java.
|
||||
|
||||
The quality of homework in this class is also unparalleled. The 14 labs will allow you to implement most of the data structures mentioned in the class by yourself, and the 10 homework will allow you to use data structures and algorithms to solve practical problems.
|
||||
|
|
@ -20,7 +22,7 @@ In addition, there are 3 projects that give you the opportunity to be exposed to
|
|||
## Resources
|
||||
## Course Resources
|
||||
|
||||
- Course Website: [spring2024](https://sp24.datastructur.es/), [fall2023](https://fa23.datastructur.es/), [spring2023](https://sp23.datastructur.es/), [spring2018](https://sp18.datastructur.es/)
|
||||
- Course Website: [spring2024](https://sp24.datastructur.es/), [fall2023](https://fa23.datastructur.es/), [spring2023](https://sp23.datastructur.es/), [spring2021](https://sp21.datastructur.es/), [spring2018](https://sp18.datastructur.es/)
|
||||
- Recordings: refer to the course website
|
||||
- Textbook: None
|
||||
- Assignments: Slightly different every year. In the spring semester of 2018, there are 14 Labs, 10 Homework and 3 Projects. Please refer to the course website for specific requirements.
|
||||
|
|
|
|||
|
|
@ -13,6 +13,8 @@
|
|||
我上的是 2018 年春季学期的版本,该课的开课老师 Josh Hug 教授慷慨地将 autograder 开源了,大家可以通过网站公开的邀请码在 [gradescope](https://gradescope.com/)
|
||||
免费加入课程,从而方便地测评自己的代码。
|
||||
|
||||
根据教授最新的政策,SP2021 的 CS61B 也对公众开放。要设置所有内容,请前往 Gradescope 并选择"Add a course"按钮。输入课程代码 **MB7ZPY** 以添加课程。
|
||||
|
||||
这门课所有的编程作业都是使用 Java 完成的。没有 Java 基础的同学也不用担心,课程会有保姆级的教程,从 IDEA(一款主流的 Java 编程环境)的配置讲起,把 Java 的核心语法与特性事无巨细地讲授,大家完全不用担心跟不上的问题。
|
||||
|
||||
这门课的作业质量也是绝绝子。14 个 lab 会让你自己实现课上所讲的绝大部分数据结构,10 个 Homework 会让你运用数据结构和算法解决实际问题,
|
||||
|
|
@ -20,7 +22,7 @@
|
|||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:[spring2024](https://sp24.datastructur.es/), [fall2023](https://fa23.datastructur.es/), [spring2023](https://sp23.datastructur.es/), [spring2018](https://sp18.datastructur.es/)
|
||||
- 课程网站:[spring2024](https://sp24.datastructur.es/), [fall2023](https://fa23.datastructur.es/), [spring2023](https://sp23.datastructur.es/), [spring2021](https://sp21.datastructur.es/), [spring2018](https://sp18.datastructur.es/)
|
||||
- 课程视频:原版视频参见课程网站,B站有中文翻译搬运。
|
||||
- 课程教材:无
|
||||
- 课程作业:每年略有不同,18 年春季学期有 14 个 Lab,10 个 Homework以及 3 个 Project,具体要求详见课程网站。
|
||||
|
|
|
|||
|
|
@ -14,9 +14,11 @@ The class is extremely friendly to novices, and Andrew has the ability to make m
|
|||
|
||||
Of course, as a public mooc, the difficulty of this course has been deliberately lowered, and many mathematical derivations are skimmed over. If you are interested in machine learning theory and want to investigate the mathematical theory behind these algorithms, you can refer to [CS229](./CS229.md) and [CS189](./CS189.md).
|
||||
|
||||
The new course is rebuilt into a specialization consisting of three courses.
|
||||
|
||||
## Course Resources
|
||||
|
||||
- Course Website: <https://www.coursera.org/learn/machine-learning>
|
||||
- Course Website: <https://www.coursera.org/specializations/machine-learning-introduction>
|
||||
- Recordings: refer to the course website
|
||||
- Textbook: None
|
||||
- Assignments: refer to the course website
|
||||
|
|
|
|||
|
|
@ -14,9 +14,11 @@
|
|||
|
||||
当然,这门课作为一个公开慕课,难度上刻意放低了些,很多数学推导大多一带而过,如果你有志于从事机器学习理论研究,想要深究这些算法背后的数学理论,可以参考 [CS229](./CS229.md) 和 [CS189](./CS189.md)。
|
||||
|
||||
新版课程更新成了一个包含三个课程的系列,可以尝试在 Coursera 申请助学金后不用订阅即可学习。
|
||||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:<https://www.coursera.org/learn/machine-learning>
|
||||
- 课程网站:<https://www.coursera.org/specializations/machine-learning-introduction>
|
||||
- 课程视频:参见课程网站
|
||||
- 课程教材:无
|
||||
- 课程作业:参见课程网站
|
||||
|
|
|
|||
|
|
@ -26,3 +26,5 @@ Instructors [Zico Kolter](https://zicokolter.com/) and [Tianqi Chen](https://tqc
|
|||
## Resource Compilation
|
||||
|
||||
All resources and assignment implementations used by @PKUFlyingPig in this course are consolidated in [PKUFlyingPig/CMU10-714 - GitHub](https://github.com/PKUFlyingPig/CMU10-714)
|
||||
|
||||
All assignment implementations by @Crazy-Ryan in this course (24 Fall offering) are consolidated in [Crazy-Ryan/CMU-10-714 - GitHub](https://github.com/Crazy-Ryan/CMU-10-714)
|
||||
|
|
@ -32,3 +32,5 @@
|
|||
## 资源汇总
|
||||
|
||||
@PKUFlyingPig 在学习这门课中用到的所有资源和作业实现都汇总在 [PKUFlyingPig/CMU10-714 - GitHub](https://github.com/PKUFlyingPig/CMU10-714) 中。
|
||||
|
||||
@Crazy-Ryan 在学习这门课(24 Fall)过程中的作业实现汇总在 [Crazy-Ryan/CMU-10-714 - GitHub](https://github.com/Crazy-Ryan/CMU-10-714) 中。
|
||||
75
docs/机器学习系统/CSE234.en.md
Normal file
75
docs/机器学习系统/CSE234.en.md
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
# CSE234: Data Systems for Machine Learning
|
||||
|
||||
## Course Overview
|
||||
|
||||
- University: UCSD
|
||||
- Prerequisites: Linear Algebra, Deep Learning, Operating Systems, Computer Networks, Distributed Systems
|
||||
- Programming Languages: Python, Triton
|
||||
- Difficulty: 🌟🌟🌟
|
||||
- Estimated Workload: ~120 hours
|
||||
|
||||
<!-- Introduce the course in one or two paragraphs, including but not limited to:
|
||||
(1) The scope of technical topics covered
|
||||
(2) Its distinguishing features compared to similar courses
|
||||
(3) Personal learning experience and impressions
|
||||
(4) Caveats and difficulty warnings for self-study
|
||||
-->
|
||||
|
||||
This course focuses on the design of end-to-end large language model (LLM) systems, serving as an introductory course to building efficient LLM systems in practice.
|
||||
|
||||
The course can be more accurately divided into three parts (with several additional guest lectures):
|
||||
|
||||
Part 1. Foundations: modern deep learning and computational representations
|
||||
|
||||
- Modern deep learning and computation graphs (framework and system fundamentals)
|
||||
- Automatic differentiation and an overview of ML system architectures
|
||||
- Tensor formats, in-depth matrix multiplication, and hardware accelerators
|
||||
|
||||
|
||||
|
||||
Part 2. Systems and performance optimization: from GPU kernels to compilation and memory
|
||||
|
||||
- GPUs and CUDA (including basic performance models)
|
||||
- GPU matrix multiplication and operator-level compilation
|
||||
- Triton programming, graph optimization, and compilation
|
||||
- Memory management (including practical issues and techniques in training and inference)
|
||||
- Quantization methods and system-level deployment
|
||||
|
||||
|
||||
Part 3. LLM systems: training and inference
|
||||
|
||||
- Parallelization strategies: model parallelism, collective communication, intra-/inter-op parallelism, and auto-parallelization
|
||||
- LLM fundamentals: Transformers, Attention, and MoE
|
||||
- LLM training optimizations (e.g., FlashAttention-style techniques)
|
||||
- LLM inference: continuous batching, paged attention, disaggregated prefill/decoding
|
||||
- Scaling laws
|
||||
|
||||
|
||||
(Guest lectures cover topics such as ML compilers, LLM pretraining and open science, fast inference, and tool use and agents, serving as complementary extensions.)
|
||||
|
||||
The defining characteristic of CSE234 is its strong focus on LLM systems as the core application setting. The course emphasizes real-world system design trade-offs and engineering constraints, rather than remaining at the level of algorithms or API usage. Assignments often require students to directly confront performance bottlenecks—such as memory bandwidth limitations, communication overheads, and kernel fusion—and address them through Triton or system-level optimizations. Overall, the learning experience is fairly intensive: a solid background in systems and parallel computing is important. For self-study, it is strongly recommended to prepare CUDA, parallel programming, and core systems knowledge in advance; otherwise, the learning curve becomes noticeably steep in the later parts of the course, especially around LLM optimization and inference. That said, once the pace is manageable, the course offers strong long-term value for those pursuing work in LLM infrastructure, ML systems, or AI compilers.
|
||||
|
||||
## Recommended Learning Path
|
||||
|
||||
The course itself is relatively well-structured and progressive. However, for students without prior experience in systems and parallel computing, the transition into the second part of the course may feel somewhat steep. A key aspect of this course is spending significant time implementing and optimizing systems in practice. Therefore, it is highly recommended to explore relevant open-source projects on GitHub while reading papers, and to implement related systems or kernels hands-on to deepen understanding.
|
||||
|
||||
- Foundations: consider studying alongside open-source projects such as [micrograd](https://github.com/karpathy/micrograd)
|
||||
- Systems & performance optimization and LLM systems: consider pairing with projects such as [nanoGPT](https://github.com/karpathy/nanoGPT) and [nano-vllm](https://github.com/GeeeekExplorer/nano-vllm)
|
||||
|
||||
The course website itself provides a curated list of additional references and materials, which can be found here:
|
||||
[Book-related documentation and courses](https://hao-ai-lab.github.io/cse234-w25/resources/#book-related-documentation-and-courses)
|
||||
|
||||
## Course Resources
|
||||
|
||||
- Course Website: https://hao-ai-lab.github.io/cse234-w25/
|
||||
- Lecture Videos: https://hao-ai-lab.github.io/cse234-w25/
|
||||
- Reading Materials: https://hao-ai-lab.github.io/cse234-w25/resources/
|
||||
- Assignments: https://hao-ai-lab.github.io/cse234-w25/assignments/
|
||||
|
||||
## Resource Summary
|
||||
|
||||
All course materials are released in open-source form. However, the online grading infrastructure and reference solutions for assignments have not been made public.
|
||||
|
||||
## Additional Resources / Further Reading
|
||||
|
||||
- [GPUMode](https://www.youtube.com/@GPUMODE): offers in-depth explanations of GPU kernels and systems. Topics referenced in the course—such as [DistServe](https://www.youtube.com/watch?v=tIPDwUepXcA), [FlashAttention](https://www.youtube.com/watch?v=VPslgC9piIw), and [Triton](https://www.youtube.com/watch?v=njgow_zaJMw)—all have excellent extended talks available.
|
||||
77
docs/机器学习系统/CSE234.md
Normal file
77
docs/机器学习系统/CSE234.md
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
# CSE234: Data Systems for Machine Learning
|
||||
|
||||
|
||||
## 课程简介
|
||||
|
||||
- 所属大学:UCSD
|
||||
- 先修要求:线性代数,深度学习,操作系统,计算机网络,分布式系统
|
||||
- 编程语言:Python, Triton
|
||||
- 课程难度:🌟🌟🌟
|
||||
- 预计学时:120小时
|
||||
|
||||
<!-- 用一两段话介绍这门课程,内容包括但不限于:
|
||||
(1)课程覆盖的知识点范围
|
||||
(2)与同类课程相比它的优势与特点
|
||||
(3)学习这门课程的体验与感受
|
||||
(4)自学这门课的注意点(踩过的坑、难度预警等等)
|
||||
(5)... ...
|
||||
-->
|
||||
|
||||
本课程专注于设计一个全面的大语言模型(LLM)系统课程,作为设计高效LLM系统的入门介绍。
|
||||
|
||||
课程可以更准确地分为三个部分(外加若干 guest lecture):
|
||||
|
||||
Part 1. 基础:现代深度学习与计算表示
|
||||
|
||||
- Modern DL 与计算图(computational graph / framework 基础)
|
||||
- Autodiff 与 ML system 架构概览
|
||||
- Tensor format、MatMul 深入与硬件加速器(accelerators)
|
||||
|
||||
|
||||
Part 2. 系统与性能优化:从 GPU Kernel 到编译与内存
|
||||
|
||||
- GPUs & CUDA(含基本性能模型)
|
||||
- GPU MatMul 与算子编译(operator compilation)
|
||||
- Triton 编程、图优化与编译(graph optimization & compilation)
|
||||
- Memory(含训练/推理中的内存问题与技巧)
|
||||
- Quantization(量化方法与系统落地)
|
||||
|
||||
|
||||
Part 3. LLM系统:训练与推理
|
||||
|
||||
- 并行策略:模型并行、collective communication、intra-/inter-op、自动并行化
|
||||
- LLM 基础:Transformer、Attention、MoE
|
||||
- LLM 训练优化:FlashAttention 等
|
||||
- LLM 推理:continuous batching、paged attention、disaggregated prefill/decoding
|
||||
- Scaling law
|
||||
|
||||
|
||||
(Guest lectures:ML compiler、LLM pretraining/open science、fast inference、tool use & agents 等,作为补充与扩展。)
|
||||
|
||||
CSE234的最大特点在于非常专注于以LLM (LLM System)为核心应用场景,强调真实系统设计中的取舍与工程约束,而非停留在算法或 API 使用层面。课程作业通常需要直接面对性能瓶颈(如内存带宽、通信开销、kernel fusion 等),并通过 Triton 或系统级优化手段加以解决,对理解“为什么某些 LLM 系统设计是现在这个样子”非常有帮助。学习体验整体偏硬核,前期对系统与并行计算背景要求较高,自学时建议提前补齐 CUDA/并行编程与基础系统知识,否则在后半部分(尤其是 LLM 优化与推理相关内容)会明显感到陡峭的学习曲线。但一旦跟上节奏,这门课对从事 LLM Infra / ML Systems / AI Compiler 方向的同学具有很强的长期价值。
|
||||
|
||||
|
||||
## 学习路线推荐
|
||||
|
||||
课程本身其实比较循序渐进,但是对于没有系统与并行计算背景的同学来说可能到第二部分会感觉稍微陡峭一点。课程最核心的部分其实是要花很多时间动手实现与优化系统,因此建议在读paper的时候就可以在Github上找一些相关的开源项目,动手实现相关的系统或者Kernel,加深理解。
|
||||
|
||||
- 基础部分:建议配合 [micrograd](https://github.com/karpathy/micrograd) 等开源项目一起学习
|
||||
- 系统与性能优化 & LLM系统:建议配合 [nanoGPT](https://github.com/karpathy/nanoGPT), [nano-vllm](https://github.com/GeeeekExplorer/nano-vllm) 等开源项目一起食用
|
||||
|
||||
课程页面本身提供了一些知识与资源,可以参考:[Book related documentation and courses](https://hao-ai-lab.github.io/cse234-w25/resources/#book-related-documentation-and-courses)
|
||||
|
||||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:https://hao-ai-lab.github.io/cse234-w25/
|
||||
- 课程视频:https://hao-ai-lab.github.io/cse234-w25/
|
||||
- 课程教材:https://hao-ai-lab.github.io/cse234-w25/resources/
|
||||
- 课程作业:https://hao-ai-lab.github.io/cse234-w25/assignments/
|
||||
|
||||
## 资源汇总
|
||||
|
||||
所有课程内容都发布了对应的开源版本,但在线测评和作业参考答案部分尚未开源。
|
||||
|
||||
## 其他资源/课程延伸
|
||||
|
||||
- [GPUMode](https://www.youtube.com/@GPUMODE): 有非常多关于GPU Kernel / System的深度讲解。课程中提到的包括[DistServe](https://www.youtube.com/watch?v=tIPDwUepXcA), [FlashAttention](https://www.youtube.com/watch?v=VPslgC9piIw), [Triton](https://www.youtube.com/watch?v=njgow_zaJMw) 都有很好的延伸
|
||||
24
docs/深度学习/CMU11-785.en.md
Normal file
24
docs/深度学习/CMU11-785.en.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# CMU 11-785: Introduction to Deep Learning
|
||||
|
||||
## Descriptions
|
||||
|
||||
- Offered by: CMU
|
||||
- Prerequisites: Linear Algebra, Probability, Python Programming, and ML Foundations
|
||||
- Programming Languages: Python
|
||||
- Difficulty: 🌟🌟🌟🌟🌟
|
||||
- Class Hour: ~120 hours
|
||||
|
||||
CMU 11-785 is a rigorous, fast-paced deep learning core course with very little filler. It starts from neural network fundamentals and systematically covers CNNs, RNNs, Attention/Transformers, optimization, and generalization.
|
||||
|
||||
The workload feels close to graduate-level training: assignments usually require real understanding of model behavior, training details, and experimental methodology. If you want durable deep learning fundamentals (instead of only using high-level APIs), this course is an excellent investment.
|
||||
|
||||
## Course Resources
|
||||
|
||||
- Course Website: <https://deeplearning.cs.cmu.edu/S26/index.html>
|
||||
- Recordings: Lecture recordings are available on course websites (varies by semester)
|
||||
- Textbooks: Mainly Lecture Notes / Slides + paper readings
|
||||
- Assignments: Multiple programming assignments and a course project (published on course sites)
|
||||
|
||||
## Personal Resources
|
||||
|
||||
No public personal repository is currently provided for this course.
|
||||
24
docs/深度学习/CMU11-785.md
Normal file
24
docs/深度学习/CMU11-785.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# CMU 11-785: Introduction to Deep Learning
|
||||
|
||||
## 课程简介
|
||||
|
||||
- 所属大学:CMU
|
||||
- 先修要求:线性代数、概率论、Python 编程、机器学习基础
|
||||
- 编程语言:Python
|
||||
- 课程难度:🌟🌟🌟🌟🌟
|
||||
- 预计学时:120 小时
|
||||
|
||||
CMU 11-785 是一门非常“硬核”的深度学习核心课,整体风格扎实、节奏快、几乎没有“水内容”。课程从神经网络基础出发,逐步覆盖 CNN、RNN、Attention/Transformer、优化与泛化等核心主题,适合想把理论与实践一起打牢的同学。
|
||||
|
||||
这门课的学习体验更接近“研究生强度训练”:作业通常不只是套模板跑通,而是要求你理解模型行为、训练细节和实验设计。若你希望建立长期可迁移的深度学习能力(而非只会调几个现成 API),这门课的投入产出比很高。
|
||||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:<https://deeplearning.cs.cmu.edu/S26/index.html>
|
||||
- 课程视频:课程网站提供 Lecture 录像(不同学期页面会更新)
|
||||
- 课程教材:以 Lecture Notes / Slides + 论文阅读为主
|
||||
- 课程作业:课程网站发布,通常包含多个编程作业与课程项目
|
||||
|
||||
## 资源汇总
|
||||
|
||||
暂未提供公开的个人课程笔记与作业仓库。
|
||||
|
|
@ -13,6 +13,6 @@ Stanford's CV introductory class, led by the giant of the computer field, Fei-Fe
|
|||
## Course Resources
|
||||
|
||||
- Course Website:<http://cs231n.stanford.edu/>
|
||||
- Course Video:<https://www.bilibili.com/video/BV1nJ411z7fe>
|
||||
- Course Video:[spring 2017 Bilibili (Classic)](https://www.bilibili.com/video/BV1nJ411z7fe), [spring 2025 YouTube (Latest)](https://www.youtube.com/playlist?list=PLoROMvodv4rOmsNzYBMe0gJY2XS8AQg16)
|
||||
- Course Materials: None
|
||||
- Coursework:<http://cs231n.stanford.edu/schedule.html>,3 Programming Assignments
|
||||
|
|
|
|||
|
|
@ -13,6 +13,6 @@ Stanford 的 CV 入门课,由计算机领域的巨佬李飞飞院士领衔教
|
|||
## 课程资源
|
||||
|
||||
- 课程网站:<http://cs231n.stanford.edu/>
|
||||
- 课程视频:<https://www.bilibili.com/video/BV1nJ411z7fe>
|
||||
- 课程视频:[spring 2017 Bilibili](https://www.bilibili.com/video/BV1nJ411z7fe), [spring 2025 YouTube (最新)](https://www.youtube.com/playlist?list=PLoROMvodv4rOmsNzYBMe0gJY2XS8AQg16)
|
||||
- 课程教材:无
|
||||
- 课程作业:<http://cs231n.stanford.edu/schedule.html>,3个编程作业
|
||||
|
|
|
|||
24
docs/深度学习/MIT6-7960.en.md
Normal file
24
docs/深度学习/MIT6-7960.en.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# MIT 6.7960: Deep Learning
|
||||
|
||||
## Descriptions
|
||||
|
||||
- Offered by: MIT
|
||||
- Prerequisites: Linear Algebra, Probability/Statistics, and ML Foundations
|
||||
- Programming Languages: Python
|
||||
- Difficulty: 🌟🌟🌟🌟
|
||||
- Class Hour: ~90 hours
|
||||
|
||||
A key strength of MIT 6.7960 is breadth. Beyond mainstream LLM/NLP topics, it also includes computer-vision-oriented content, making it a good choice if you want broader coverage rather than a single-track focus.
|
||||
|
||||
The course typically combines theory, model design, and application cases. For self-learners with basic foundations, its assignments/projects are practical for advanced training across multiple subfields.
|
||||
|
||||
## Course Resources
|
||||
|
||||
- Course Website: <https://ocw.mit.edu/courses/6-7960-deep-learning-fall-2024/>
|
||||
- Recordings: Lecture videos and related materials on MIT OCW
|
||||
- Textbooks: Lecture Notes + recommended papers
|
||||
- Assignments: Assignment and project information on MIT OCW
|
||||
|
||||
## Personal Resources
|
||||
|
||||
No public personal repository is currently provided for this course.
|
||||
24
docs/深度学习/MIT6-7960.md
Normal file
24
docs/深度学习/MIT6-7960.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# MIT 6.7960: Deep Learning
|
||||
|
||||
## 课程简介
|
||||
|
||||
- 所属大学:MIT
|
||||
- 先修要求:线性代数、概率统计、机器学习基础
|
||||
- 编程语言:Python
|
||||
- 课程难度:🌟🌟🌟🌟
|
||||
- 预计学时:90 小时
|
||||
|
||||
MIT 6.7960 的优势在于覆盖面相对更广:除了主流的 LLM/NLP 主线,也会涉及计算机视觉等任务与方法,适合希望建立“多方向通识”的学习者,而不是只押注单一赛道。
|
||||
|
||||
课程通常会把理论、模型设计和应用案例结合起来,帮助你理解“同一套深度学习思想”如何在不同任务中落地。对自学者来说,这门课的作业和项目也很适合做进阶练手。
|
||||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:<https://ocw.mit.edu/courses/6-7960-deep-learning-fall-2024/>
|
||||
- 课程视频:MIT OCW 页面提供 Lecture 视频与相关材料
|
||||
- 课程教材:Lecture Notes / 讲义 + 推荐论文
|
||||
- 课程作业:MIT OCW 页面提供作业与项目信息
|
||||
|
||||
## 资源汇总
|
||||
|
||||
暂未提供公开的个人课程笔记与作业仓库。
|
||||
24
docs/深度学习/NYU-DLSP21.en.md
Normal file
24
docs/深度学习/NYU-DLSP21.en.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# NYU DLSP21: NYU Deep Learning Spring 2021
|
||||
|
||||
## Descriptions
|
||||
|
||||
- Offered by: NYU
|
||||
- Prerequisites: Linear Algebra, Probability, and Python Programming
|
||||
- Programming Languages: Python
|
||||
- Difficulty: 🌟🌟🌟🌟
|
||||
- Class Hour: ~80 hours
|
||||
|
||||
The most distinctive feature of this course is that it is taught by Yann LeCun. For many learners, this is a rare public opportunity to follow a full deep learning course from a top researcher.
|
||||
|
||||
Its style balances theory and intuition with a research-oriented perspective. It works especially well as a high-quality follow-up after introductory courses, helping you build better instincts for problem framing and model design.
|
||||
|
||||
## Course Resources
|
||||
|
||||
- Course Website: <https://atcold.github.io/NYU-DLSP21/>
|
||||
- Recordings: Full lecture videos on the course website
|
||||
- Textbooks: Lecture Notes / Slides
|
||||
- Assignments: Assignments and practical materials on the course website
|
||||
|
||||
## Personal Resources
|
||||
|
||||
No public personal repository is currently provided for this course.
|
||||
24
docs/深度学习/NYU-DLSP21.md
Normal file
24
docs/深度学习/NYU-DLSP21.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# NYU DLSP21: NYU Deep Learning Spring 2021
|
||||
|
||||
## 课程简介
|
||||
|
||||
- 所属大学:NYU
|
||||
- 先修要求:线性代数、概率论、Python 编程
|
||||
- 编程语言:Python
|
||||
- 课程难度:🌟🌟🌟🌟
|
||||
- 预计学时:80 小时
|
||||
|
||||
这门课最具辨识度的亮点是由 Yann LeCun 亲自授课。对很多同学而言,这是极少见的公开系统课程资源:可以完整地听到顶级学者从一线研究者视角讲解深度学习。
|
||||
|
||||
课程讲解兼顾理论与直觉,整体偏研究导向,适合已完成入门课后继续提升“问题意识”和“建模品味”。把它作为主线课程之外的高质量补充,会非常有价值。
|
||||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:<https://atcold.github.io/NYU-DLSP21/>
|
||||
- 课程视频:课程网站提供完整 Lecture 视频
|
||||
- 课程教材:Lecture Notes / Slides
|
||||
- 课程作业:课程网站提供作业与实验材料
|
||||
|
||||
## 资源汇总
|
||||
|
||||
暂未提供公开的个人课程笔记与作业仓库。
|
||||
20
docs/深度生成模型/MIT6.S184.en.md
Normal file
20
docs/深度生成模型/MIT6.S184.en.md
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
# MIT6.S184: Generative AI with Stochastic Differential Equations
|
||||
|
||||
## Course Introduction
|
||||
|
||||
- University: MIT
|
||||
- Prerequisites: Basic understanding of deep learning, and be comfortable with calculus and linear algebra
|
||||
- Programming Language: Python (with PyTorch)
|
||||
- Course Difficulty: 🌟🌟🌟🌟
|
||||
- Estimated Study Hours: 20
|
||||
|
||||
This course is an introductory diffusion model course offered during MIT's IAP term by MIT CSAIL. Taught by MIT students Peter Holderrieth and Ezra Erives, the course provides a clear and accessible explanation of the mathematical foundations of diffusion and flow-matching models from the perspective of differential equations. It also includes hands-on labs where students build diffusion models from scratch, concluding with lectures on applications in cutting-edge areas such as molecular design and robotics.
|
||||
|
||||
The accompanying lecture notes are exceptionally well-written and highly recommended for in-depth reading.
|
||||
|
||||
## Course Resources
|
||||
|
||||
- Course Website: https://diffusion.csail.mit.edu/
|
||||
- Course Videos: See course website
|
||||
- Course Textbook: [An Introduction to Flow Matching and Diffusion Models](https://arxiv.org/abs/2506.02070)
|
||||
- Course Assignments: Three labs, see course website for details
|
||||
20
docs/深度生成模型/MIT6.S184.md
Normal file
20
docs/深度生成模型/MIT6.S184.md
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
# MIT6.S184: Generative AI with Stochastic Differential Equations
|
||||
|
||||
## 课程简介
|
||||
|
||||
- 所属大学:MIT
|
||||
- 先修要求:Basic understanding of deep learning, and be comfortable with calculus and linear algebra
|
||||
- 编程语言:Python (with PyTorch)
|
||||
- 课程难度:🌟🌟🌟🌟
|
||||
- 预计学时:20
|
||||
|
||||
这门课程是由 MIT CSAIL 的 IAP 小学期开办的扩散模型入门课程。该课程由 MIT 学生 Peter Holderrieth 和 Ezra Erives 主讲,从微分方程的视角深入浅出地讲解了扩散模型和流匹配模型的数学理论基础,并且配以实践让学生从零构建扩散模型,最后通过讲座介绍其在分子设计和机器人学等前沿技术中的应用。
|
||||
|
||||
课程配套的教材笔记写得非常好,推荐仔细阅读。
|
||||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:https://diffusion.csail.mit.edu/
|
||||
- 课程视频:参见课程网站
|
||||
- 课程教材:[An Introduction to Flow Matching and Diffusion Models](https://arxiv.org/abs/2506.02070)
|
||||
- 课程作业:三个实验,具体参见课程网站
|
||||
|
|
@ -6,6 +6,8 @@ In fact, LLMs are just one branch of deep generative models. Other types such as
|
|||
|
||||
Recommended courses for learning:
|
||||
|
||||
- [MIT 6.S184: Generative AI with Stochastic Differential Equations](./MIT6.S184.md): An introductory GenAI course offered during MIT's IAP term. It explains the mathematical foundations behind Flow Matching and Diffusion Models from the perspective of differential equations, accompanied by simple hands-on labs to help students grasp the concepts through practice. Ideal for those interested in the underlying mathematical principles.
|
||||
|
||||
- [MIT 6.S978: Deep Generative Models](https://mit-6s978.github.io/schedule.html): Taught by MIT’s rising star Prof. Kaiming He, this course covers fundamental theories and cutting-edge papers related to various generative models. The assignments include well-prepared scaffold code. While not overly difficult, they help deepen understanding and provide a quick, comprehensive view of the field.
|
||||
|
||||
- [UCB CS294-158-SP24: Deep Unsupervised Learning](https://sites.google.com/view/berkeley-cs294-158-sp24/home): Taught by reinforcement learning giant Pieter Abbeel. Compared to the MIT course, this one is more comprehensive and includes lecture videos and slides. The homework only provides test code, so students must implement model architecture and training code themselves. Though demanding, it’s ideal for those who want hands-on experience in training models. As is well known, there are many practical tricks in deep learning, and the devil is often in the details. Nothing teaches those details better than training a model yourself.
|
||||
|
|
@ -16,6 +18,8 @@ The GPT series by OpenAI has demonstrated remarkable performance under the guida
|
|||
|
||||
- [Stanford CS336: Language Modeling from Scratch](https://stanford-cs336.github.io/spring2025/index.html): As the title suggests, this course teaches you to build all the core components of an LLM from scratch, such as the tokenizer, model architecture, training optimizer, low-level operators, data cleaning, and post-training algorithms. Each assignment has a 40-50 page PDF handout—very rigorous. Highly recommended if you want to fully understand every low-level detail of LLMs.
|
||||
|
||||
- [CMU 15-779: Advanced Topics in Machine Learning Systems (LLM Edition)](./大语言模型/CMU15-779.md) / [EN](./大语言模型/CMU15-779.en.md): A systems and performance-oriented course that explains how high-level models are decomposed into low-level kernels and executed efficiently on heterogeneous accelerators and in distributed environments. Topics include CUDA, ML compilation, graph-level optimizations, auto-parallelization, and LLM serving/inference acceleration, along with weekly paper reviews and a final systems project.
|
||||
|
||||
- [CMU 11868: Large Language Model Systems](https://llmsystem.github.io/llmsystem2025spring/): This CMU course focuses on system-level optimization of LLMs, including GPU acceleration, distributed training/inference, and cutting-edge techniques. Great for students in systems research to gain a holistic understanding of the field. (Disclosure: One of my papers on PD decoupling is included in the syllabus, hence the personal recommendation.) Assignments involve implementing a mini-PyTorch framework and then building system-level LLM optimizations on top of it.
|
||||
|
||||
- [CMU 11667: Large Language Models: Methods and Applications](https://cmu-llms.org/) and [CMU 11711: Advanced NLP](https://www.phontron.com/class/anlp-fall2024/): Compared to the previous two, these courses focus more on higher-level algorithms and applications. Each lecture includes many recommended readings, making them suitable for gaining a broad understanding of LLM research frontiers. You can then dive deeper into any subfield that interests you based on the reading materials.
|
||||
|
|
|
|||
|
|
@ -5,6 +5,8 @@
|
|||
其实,大语言模型只是深度生成模型的一个分支,而其他生成模型例如 VAE,GAN,Diffusion Model,Flow 等等,都还在“生成”这一领域占有重要地位,所谓的 AIGC,就是泛指这一类技术。
|
||||
推荐学习下列课程:
|
||||
|
||||
- [MIT 6.S184: Generative AI with Stochastic Differential Equations](./MIT6.S184.md): MIT IAP 小学期的 GenAI 入门课程,主要通过微分方程的视角讲解了 Flow Matching 和 Diffusion Model 背后的数学原理,并且配有简单的小实验让学生在实践中理解,适合对底层数学原理感兴趣的同学入门。
|
||||
|
||||
- [MIT 6.S978: Deep Generative Models](https://mit-6s978.github.io/schedule.html): MIT 新晋明星教授何恺明亲授,涵盖了各种生成模型的基础理论和相关前沿论文,几次作业都有丰富的脚手架代码,难度不高但能加深理解,能对这个领域有个快速全貌了解。
|
||||
|
||||
- [UCB CS294-158-SP24: Deep Unsupervised Learning](https://sites.google.com/view/berkeley-cs294-158-sp24/home): 强化学习领域的顶级巨佬 Pieter Abbeel 主讲,相比 MIT 的课程内容更加丰富全面,并且有配套课程视频和 Slides。此外课后作业只有测试代码,需要学生自主编写模型架构定义和训练代码,虽然硬核但很适合有志于炼丹的同学练手。众所周知,深度学习理论实践中存在着很多经验技巧,魔鬼往往存在于细节里。没有什么比自己上手训一个模型更能掌握这些细节了。
|
||||
|
|
@ -16,6 +18,8 @@ OpenAI 的 GPT 系列让大语言模型在 Scaling Law 的指引下展现出惊
|
|||
|
||||
- [Stanford CS336: Language Modeling from Scratch](https://stanford-cs336.github.io/spring2025/index.html): 正如课程标题写的,在这门课程中你将从头编写大语言模型的所有核心组件,例如 Tokenizer,模型架构,训练优化器,底层算子,训练数据清洗,后训练算法等等。每次作业的 handout 都有四五十页 pdf,相当硬核。如果你想充分吃透大语言模型的所有底层细节,那么非常推荐学习这门课程。
|
||||
|
||||
- [CMU 15-779: Advanced Topics in Machine Learning Systems (LLM Edition)](./大语言模型/CMU15-779.md) / [EN](./大语言模型/CMU15-779.en.md): 偏系统与性能优化视角,重点讲清“高层模型如何分解成 kernel 并在异构加速器与分布式环境中高效执行”,覆盖 CUDA、ML 编译、图级优化、自动并行化、LLM Serving 与推理加速等内容,并配套按周论文阅读与期末系统项目。
|
||||
|
||||
- [CMU 11868: Large Language Model Systems](https://llmsystem.github.io/llmsystem2025spring/): CMU 的大语言模型系统课程,侧重底层系统优化,例如 GPU 加速,分布式训练和推理,以及各种前沿技术。非常适合从事系统领域的同学对这个方向有个全貌性的了解。课表里还包含了一篇我发表的 PD 分离相关的文章,因此私心推荐一下。课程作业的话会让你先实现一个迷你 Pytorch,然后在上面实现各种大语言模型的系统级优化。
|
||||
|
||||
- [CMU 11667: Large Language Models: Methods and Applications](https://cmu-llms.org/) 和 [CMU 11711: Advanced NLP](https://www.phontron.com/class/anlp-fall2024/): 和前两门课相比,这两门课更偏重上层算法和应用,而且每节课都列举了很多相关阅读材料,适合对大语言模型发展前沿的各个方向都有个粗糙的认识,如果对某个子领域感兴趣的话再寻着参考资料深入学习。
|
||||
|
|
|
|||
35
docs/深度生成模型/大语言模型/CMU15-779.en.md
Normal file
35
docs/深度生成模型/大语言模型/CMU15-779.en.md
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
# CMU 15-779: Advanced Topics in Machine Learning Systems (LLM Edition)
|
||||
|
||||
## Course Overview
|
||||
|
||||
- University: Carnegie Mellon University
|
||||
- Prerequisites: No strict prerequisites; an intro ML background and hands-on deep learning training experience are recommended; familiarity with PyTorch helps; basic CUDA/GPU knowledge will significantly improve the learning curve
|
||||
- Programming Language: Python (systems and kernel-level topics involve CUDA/hardware concepts)
|
||||
- Course Difficulty: 🌟🌟🌟🌟
|
||||
- Estimated Study Hours: 80-120 hours
|
||||
|
||||
This course takes a systems-first view of modern machine learning and LLM infrastructure. The core question it repeatedly answers is: how does a model written in a high-level framework (e.g., PyTorch) get decomposed into low-level kernels, and how is it executed efficiently on heterogeneous accelerators (GPUs/TPUs) and in distributed environments. The syllabus covers GPU programming, ML compilers, graph-level optimizations, distributed training and auto-parallelization, and LLM serving and inference acceleration. It is a strong fit if you want to connect “framework-level experience” with “kernels, compilation, hardware, and cluster execution.”
|
||||
|
||||
The workload is organized around consistent pre-lecture reading assignments (paper reviews) and a team-based final course project (proposal, presentation, report). For self-study, it is best to follow the schedule week by week rather than treating it as a slide-only course.
|
||||
|
||||
## Topics Covered
|
||||
|
||||
The course is structured as lectures, with major themes including:
|
||||
|
||||
1. ML systems fundamentals via TensorFlow/PyTorch (abstractions, execution models)
|
||||
2. GPU architecture and CUDA programming (memory, performance tuning)
|
||||
3. Transformer and attention case studies (FlashAttention and IO-aware attention)
|
||||
4. Advanced CUDA techniques (warp specialization, mega kernels)
|
||||
5. ML compilation (tile-based DSLs like Triton, kernel auto-tuning, graph-level optimizations, superoptimization such as Mirage)
|
||||
6. Parallelization and distributed training (ZeRO/FSDP, model/pipeline parallelism, auto-parallelization such as Alpa)
|
||||
7. LLM serving and inference (batching, PagedAttention, RadixAttention, speculative decoding)
|
||||
8. Post-training and architectures (PEFT like LoRA/QLoRA, MoE architectures/kernels/parallelism)
|
||||
|
||||
## Course Resources
|
||||
|
||||
- Course Website: <https://www.cs.cmu.edu/~zhihaoj2/15-779/>
|
||||
- Schedule (slides and reading list per lecture): <https://www.cs.cmu.edu/~zhihaoj2/15-779/schedule.html>
|
||||
- Slides (PDF): <https://www.cs.cmu.edu/~zhihaoj2/15-779/slides/>
|
||||
- Logistics (grading, paper reviews, course project): <https://www.cs.cmu.edu/~zhihaoj2/15-779/logistics.html>
|
||||
- Materials (intro deep learning materials): <https://www.cs.cmu.edu/~zhihaoj2/15-779/materials.html>
|
||||
|
||||
35
docs/深度生成模型/大语言模型/CMU15-779.md
Normal file
35
docs/深度生成模型/大语言模型/CMU15-779.md
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
# CMU 15-779: Advanced Topics in Machine Learning Systems (LLM Edition)
|
||||
|
||||
## 课程简介
|
||||
|
||||
- 所属大学:Carnegie Mellon University
|
||||
- 先修要求:无硬性先修要求;建议具备机器学习入门与深度学习训练经验,熟悉 PyTorch;了解 CUDA/GPU 基础会显著提升学习效率
|
||||
- 编程语言:Python(系统与算子层面内容可能涉及 CUDA/硬件概念)
|
||||
- 课程难度:4/5
|
||||
- 预计学时:80-120 学时
|
||||
|
||||
这门课从系统视角系统性回答一个核心问题:一个用高层框架(例如 PyTorch)写出来的模型,是如何被分解为底层 kernel,并在异构硬件加速器(GPU/TPU)与分布式环境中高效执行的。课程覆盖 GPU 编程、ML 编译器、图级优化、分布式训练与自动并行化、LLM Serving 与推理加速等主题,强系统导向,适合希望把“框架层经验”向“算子/编译/硬件/集群执行”打通的人。
|
||||
|
||||
从教学组织上,这门课会要求你持续完成课前论文阅读(paper review / reading assignments),并以小组形式完成期末系统类课程项目(proposal、presentation、report),因此自学时建议把它当成一个“按周推进的系统训练营”,而不是只看几份 slide。
|
||||
|
||||
## 课程内容
|
||||
|
||||
课程内容以 lecture 为主线,主题大致包括:
|
||||
|
||||
1. ML 系统基础:以 TensorFlow/PyTorch 为例理解计算图、执行模型与系统抽象
|
||||
2. GPU 架构与 CUDA 编程:硬件与编程模型、内存与性能优化要点
|
||||
3. Transformer 与 Attention 案例:FlashAttention 等 IO-aware attention 优化思路
|
||||
4. 高级 CUDA 编程:warp specialization、mega kernel 等低延迟/高吞吐优化技术
|
||||
5. ML 编译:Tile-based DSL(Triton 等)、内核自动调优(Ansor 等)、图级优化(TASO/PET 等)、超优化(Mirage)
|
||||
6. 并行化与分布式训练:ZeRO/FSDP、模型/流水线并行、自动并行化(Alpa 等)
|
||||
7. LLM 推理与服务:批处理、PagedAttention、RadixAttention、推测解码等
|
||||
8. 后训练与模型结构:参数高效微调(LoRA/QLoRA)、MoE(架构、kernel、并行化)
|
||||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:<https://www.cs.cmu.edu/~zhihaoj2/15-779/>
|
||||
- 课程安排(含每讲 slide 与阅读列表):<https://www.cs.cmu.edu/~zhihaoj2/15-779/schedule.html>
|
||||
- 课程讲义(PDF slides):<https://www.cs.cmu.edu/~zhihaoj2/15-779/slides/>
|
||||
- 课程规则与项目要求(Grading、Paper Review、Course Project):<https://www.cs.cmu.edu/~zhihaoj2/15-779/logistics.html>
|
||||
- 预备材料(深度学习入门材料汇总):<https://www.cs.cmu.edu/~zhihaoj2/15-779/materials.html>
|
||||
|
||||
|
|
@ -13,5 +13,5 @@ Just as the course name indicated, this course will teach the missing things in
|
|||
## Course Resources
|
||||
|
||||
- Course Website: <https://missing.csail.mit.edu/>
|
||||
- Recordings: <https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS43e29ro7I57J>
|
||||
- Recordings: [IAP 2020](https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS43e29ro7I57J), [IAP 2026](https://www.youtube.com/playlist?list=PLyzOVJj3bHQunmnnTXrNbZnBaCA-ieK4L) in YouTube
|
||||
- Assignments: Some exercises after each lecture, refer to the course website.
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@
|
|||
|
||||
- 课程网站:<https://missing.csail.mit.edu/2020/>
|
||||
- 课程中文网站: <https://missing-semester-cn.github.io/>
|
||||
- 课程视频:<https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS43e29ro7I57J>
|
||||
- 课程视频:[IAP 2020](https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS43e29ro7I57J), [IAP 2026](https://www.youtube.com/playlist?list=PLyzOVJj3bHQunmnnTXrNbZnBaCA-ieK4L) in YouTube
|
||||
- 课程中文字幕视频:
|
||||
- Missing_Semi_中译组(未完结):<https://space.bilibili.com/1010983811?spm_id_from=333.337.search-card.all.click>
|
||||
- 刘黑黑a(已完结):<https://space.bilibili.com/518734451?spm_id_from=333.337.search-card.all.click>
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
- Difficulty: 🌟🌟🌟
|
||||
- Class Hour: 40 hours
|
||||
|
||||
[Jeehoon Kang](https://cp.kaist.ac.kr/jeehoon.kang) from KAIST and his [Concurrency and Parallelism Laboratory](https://cp.kaist.ac.kr/) appear to be strong advocates of the Rust programming language. Their contributions include [CS431](https://csdiy.wiki/%E7%BC%96%E7%A8%8B%E5%85%A5%E9%97%A8/Rust/cs431/) and [CS420](https://csdiy.wiki/%E7%BC%96%E8%AF%91%E5%8E%9F%E7%90%86/CS420/) in the csidy curriculum. Naturally, they have developed an introductory course for Rust, which is this course. It covers most of the essential topics needed to get started with Rust.
|
||||
[Jeehoon Kang](https://cp.kaist.ac.kr/jeehoon.kang) from KAIST and his [Concurrency and Parallelism Laboratory](https://cp.kaist.ac.kr/) appear to be strong advocates of the Rust programming language. Their contributions include [CS431](https://csdiy.wiki/%E7%BC%96%E7%A8%8B%E5%85%A5%E9%97%A8/Rust/cs431/) and [CS420](https://csdiy.wiki/%E7%BC%96%E8%AF%91%E5%8E%9F%E7%90%86/CS420/) in the csdiy curriculum. Naturally, they have developed an introductory course for Rust, which is this course. It covers most of the essential topics needed to get started with Rust.
|
||||
|
||||
This course does not have an official textbook. The course homepage recommends using the [Rust book](https://doc.rust-lang.org/book/) for learning and provides a structured learning path in the [slides](https://docs.google.com/presentation/d/17G3SwkE_tq0H3lTt9N0ysIbHhqDZBfHkoWD5LwwAKSo/edit#slide=id.p). Although there are no publicly available lecture videos, the comprehensive testing system makes this course an excellent resource for practicing Rust. Some exercises can serve as a great supplement to [CS110L](https://csdiy.wiki/%E7%BC%96%E7%A8%8B%E5%85%A5%E9%97%A8/Rust/CS110L/). If you still feel the need for more practice after completing CS110L, this course is a good choice. Some exercises are quite challenging, and Jeehoon Kang encourages the use of AI-assisted programming. However, AI is not perfect, and the core work must still be done by yourself.
|
||||
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
- 课程难度:🌟🌟🌟
|
||||
- 预计学时:40 小时
|
||||
|
||||
来自 KAIST 的 [Jeehoon Kang](https://cp.kaist.ac.kr/jeehoon.kang) 以及他所领导的 [Concurrency and Parallelism Laboratory](https://cp.kaist.ac.kr/) 实验室似乎是 Rust 语言的忠实拥趸,csidy 之中的 [CS431](https://csdiy.wiki/%E7%BC%96%E7%A8%8B%E5%85%A5%E9%97%A8/Rust/cs431/) 和 [CS420](https://csdiy.wiki/%E7%BC%96%E8%AF%91%E5%8E%9F%E7%90%86/CS420/) 都是他们的杰作。自然,他们肯定会开发一款针对 Rust 的入门课程,也就是本课程。课程涵盖了 Rust 入门所需的绝大多数知识点。
|
||||
来自 KAIST 的 [Jeehoon Kang](https://cp.kaist.ac.kr/jeehoon.kang) 以及他所领导的 [Concurrency and Parallelism Laboratory](https://cp.kaist.ac.kr/) 实验室似乎是 Rust 语言的忠实拥趸,csdiy 之中的 [CS431](https://csdiy.wiki/%E7%BC%96%E7%A8%8B%E5%85%A5%E9%97%A8/Rust/cs431/) 和 [CS420](https://csdiy.wiki/%E7%BC%96%E8%AF%91%E5%8E%9F%E7%90%86/CS420/) 都是他们的杰作。自然,他们肯定会开发一款针对 Rust 的入门课程,也就是本课程。课程涵盖了 Rust 入门所需的绝大多数知识点。
|
||||
|
||||
本课没有指定官方教材,课程主页推荐采用 [Rust book](https://doc.rust-lang.org/book/) 学习,并在 [slides](https://docs.google.com/presentation/d/17G3SwkE_tq0H3lTt9N0ysIbHhqDZBfHkoWD5LwwAKSo/edit#slide=id.p) 之中规划了大致的学习线路。虽然没有公开课程教学视频,不过完善的测试系统仍然可以使这门课作为 Rust 习题课来练手,部分习题可以作为 [CS110L](https://csdiy.wiki/%E7%BC%96%E7%A8%8B%E5%85%A5%E9%97%A8/Rust/CS110L/) 的良好补充。如果在学习完 CS110L 之后仍然觉得需要更多练习,可以选择本课程。部分习题具有一定难度,Jeehoon Kang 对使用 AI 辅助编程持有鼓励态度,但是 AI 并不完美,核心工作仍需自己完成。
|
||||
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ I personally feel that the assignments are difficult, but the gains are great. W
|
|||
|
||||
## Course Resources
|
||||
|
||||
- Course Website: [https://stanford-cs242.github.io/f19/](https://stanford-cs242.github.io/f19/)
|
||||
- Course Website: [Latest](https://web.stanford.edu/class/cs242/) [2019 Fall](https://stanford-cs242.github.io/f19/)
|
||||
- Recordings: None, the main learning resources are the course notes and assignments
|
||||
- Textbooks: The first half of the course is based on the famous [TAPL](https://www.cis.upenn.edu/~bcpierce/tapl/), and the second half has no fixed textbook
|
||||
- Assignments: [Assignment Guide](https://stanford-cs242.github.io/f19/assignments/) and [Assignment Repository](https://github.com/stanford-cs242/f19-assignments)
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ CS242是一门讲程序语言 (Programming Language, PL) 的课程,但不是
|
|||
|
||||
## 课程资源
|
||||
|
||||
- 课程网站:[官网](https://stanford-cs242.github.io/f19/)
|
||||
- 课程网站:[官网](https://web.stanford.edu/class/cs242/) [2019秋](https://stanford-cs242.github.io/f19/)
|
||||
- 课程视频:无,主要学习途径是课程笔记和完成作业
|
||||
- 课程教材:前半部分是著名的[TAPL](https://www.cis.upenn.edu/~bcpierce/tapl/),后半部分则无固定教材
|
||||
- 课程作业:[实验指导](https://stanford-cs242.github.io/f19/assignments/)与[作业仓库](https://github.com/stanford-cs242/f19-assignments)
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ The biggest takeaway is that there is no longer a sense of difficulty or uncerta
|
|||
## Course Resources
|
||||
|
||||
- Course Website: <http://docs.compilers.cpl.icu/>
|
||||
- Recordings: <https://space.bilibili.com/479141149/channel/collectiondetail?sid=2312309>
|
||||
- Recordings: <https://space.bilibili.com/479141149/lists/2312309>
|
||||
- Textbook: Compilers: Principles, Techniques and Tools (Dragon Book)
|
||||
- Assignments: 10 written assignments + 8~10 programming labs
|
||||
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@ ANTLR 4 是 LL 解析器生成器,比起 LR 和 LALR 解析器生成器,它
|
|||
## 课程资源
|
||||
|
||||
- 课程网站:<http://docs.compilers.cpl.icu/>
|
||||
- 课程视频:<https://space.bilibili.com/479141149/channel/collectiondetail?sid=2312309>
|
||||
- 课程视频:<https://space.bilibili.com/479141149/lists/2312309>
|
||||
- 课程教材:龙书等
|
||||
- 课程作业:10 个书面作业 + 8 ~ 10 个编程作业带你实现一个编译器
|
||||
|
||||
|
|
|
|||
|
|
@ -10,20 +10,21 @@
|
|||
|
||||
The Compiler Principles course at Shanghai Jiao Tong University aims to implement a compiler for the Tiger language. In this course, you will learn about lexical analysis, grammar analysis, semantic analysis, escape analysis, activation records (stack frames), LLVM IR, liveness analysis, register allocation, garbage collection, object-oriented programming, functional programming, and many other topics. Similar to the Compiler Principles course at Peking University, this course offers you a great deal of freedom. The test programs will only check the correctness of the assembly code you output and will not impose any restrictions on the specific design of your compiler. You will need to build your own compiler step by step from scratch.
|
||||
|
||||
In this course, you will learn how to use frameworks such as `flexc++`, `Bisonc++`, and `LLVM`, and enhance your debugging skills through practice. The theoretical part of the course is taught by teachers from the IPADS Laboratory at Shanghai Jiao Tong University.
|
||||
In this course, you will learn how to use frameworks such as `Flex`, `Bison`, and `LLVM`, and enhance your debugging skills through practice. The theoretical part of the course is taught by teachers from the IPADS Laboratory at Shanghai Jiao Tong University.
|
||||
|
||||
## Course Resources
|
||||
|
||||
- Course Website: <https://ipads.se.sjtu.edu.cn/courses/compilers/index.shtml>
|
||||
- Slides: See the course website
|
||||
- Framework Code: See GitHub
|
||||
- Course Textbook: "Tiger Book" (Modern Compiler Implementation in C)
|
||||
- 2 Quizzes + 6 Labs
|
||||
- Course Website: <https://ipads.se.sjtu.edu.cn/courses/compilers>
|
||||
- Slides: <https://ipads.se.sjtu.edu.cn/courses/compilers/Schedule.html>
|
||||
- Framework Code: Not yet open-sourced. Old version: <https://gitee.com/east-china-normal-university_ttb_cs/tiger-compiler-25sp>; For the newest version, please contact to the TA.
|
||||
- Course Textbook: "Tiger Book" (Modern Compiler Implementation in C) <https://ipads.se.sjtu.edu.cn/courses/compilers/textbook/TigerBook-English.pdf>
|
||||
- 2 Quizzes + 5 essential labs + 2 optional labs
|
||||
- Lab 1: Straight-line Program Interpreter
|
||||
- Lab 2: Lexical Analysis
|
||||
- Lab 3: Parsing
|
||||
- Lab 2: Lexer
|
||||
- Lab 3: Parser
|
||||
- Lab 4: Type Checking
|
||||
- Lab 5
|
||||
- Part 1: Escape Analysis and Translation
|
||||
- Part 2: Tiger Compiler without Register Allocation
|
||||
- Lab 6: Register Allocation
|
||||
- Part 1: Escape Analysis
|
||||
- Part 2: Translate to LLVM
|
||||
- (Optional) Lab 6: Code Generation
|
||||
- (Optional) Lab 7: Register Allocation
|
||||
|
|
|
|||
|
|
@ -8,9 +8,9 @@
|
|||
- 课程难度:🌟🌟🌟🌟
|
||||
- 预计学时:150 小时
|
||||
|
||||
上海交通大学的编译原理课程旨在实现一个 Tiger 语言的编译器。在这门课上你可以学习到词法分析、文法分析、予以分析、逃逸分析、活动记录(栈帧)、LLVM IR、活跃分析、寄存器分配、垃圾收集、面向对象、函数式程序等众多话题。和北大的编译原理课程相似,该课程给予了你极大的自由度,测试程序只会对你输出的汇编代码的正确性进行测试,而不会对你编译器的具体设计做任何限制。你需要从一个个空文件中一步步构建出属于你自己的编译器。
|
||||
上海交通大学的编译原理课程旨在实现一个 Tiger 语言的编译器。在这门课上你可以学习到词法分析、文法分析、语义分析、逃逸分析、活动记录(栈帧)、LLVM IR、活跃分析、寄存器分配、垃圾收集、面向对象、函数式程序等众多话题。和北大的编译原理课程相似,该课程给予了你极大的自由度,测试程序只会对你输出的汇编代码的正确性进行测试,而不会对你编译器的具体设计做任何限制。你需要从一个个空文件中一步步构建出属于你自己的编译器。
|
||||
|
||||
在这门课上你将学到`flexc++`、`Bisonc++`、`LLVM`等框架的使用方法,并在练习过程中加强自己的调试能力。
|
||||
在这门课上你将学到 `Flex`、`Bison`、`LLVM` 等框架的使用方法,并在练习过程中加强自己的调试能力。
|
||||
|
||||
理论部分由上海交通大学 IPADS 实验室的老师讲述。
|
||||
|
||||
|
|
@ -18,16 +18,17 @@
|
|||
## 课程资源
|
||||
|
||||
|
||||
- 课程网站:<https://ipads.se.sjtu.edu.cn/courses/compilers/index.shtml>
|
||||
- 课件:参见课程网站
|
||||
- 框架代码:参见 GitHub
|
||||
- 课程教材:虎书(Modern Compiler Implementation in C)
|
||||
- 2 次Quiz + 6 个Lab
|
||||
- 课程网站:<https://ipads.se.sjtu.edu.cn/courses/compilers>
|
||||
- 课件:<https://ipads.se.sjtu.edu.cn/courses/compilers/Schedule.html>
|
||||
- 框架代码:暂不开源,旧版本可以见 <https://gitee.com/east-china-normal-university_ttb_cs/tiger-compiler-25sp>,新版本可以邮件联系当届助教。
|
||||
- 课程教材:虎书(Modern Compiler Implementation in C)<https://ipads.se.sjtu.edu.cn/courses/compilers/textbook/TigerBook-Chinese.pdf>
|
||||
- 2 次 Quiz + 5 个必做 Lab + 2 个选做 Lab
|
||||
- Lab 1: Straight-line Program Interpreter
|
||||
- Lab 2: Lexical Analysis
|
||||
- Lab 3: Parsing
|
||||
- Lab 2: Lexer
|
||||
- Lab 3: Parser
|
||||
- Lab 4: Type Checking
|
||||
- Lab 5
|
||||
- Part 1: Escape Analysis and Translation
|
||||
- Part 2: Tiger Compiler without register allocation
|
||||
- Lab 6: Register Allocation
|
||||
- Part 1: Escape Analysis
|
||||
- Part 2: Translate to LLVM
|
||||
- (Optional) Lab 6: Code Generation
|
||||
- (Optional) Lab 7: Register Allocation
|
||||
|
|
|
|||
|
|
@ -213,7 +213,7 @@ nav:
|
|||
- "ETHz: Computer Architecture": "体系结构/CA.md"
|
||||
- 操作系统:
|
||||
- "MIT 6.S081: Operating System Engineering": "操作系统/MIT6.S081.md"
|
||||
- "UCB CS162: Operating System": "操作系统/CS162.md"
|
||||
- "UCB CS162: Operating Systems and Systems Programming": "操作系统/CS162.md"
|
||||
- "NJU OS: Operating System Design and Implementation": "操作系统/NJUOS.md"
|
||||
- "HIT OS: Operating System": "操作系统/HITOS.md"
|
||||
- 并行与分布式系统:
|
||||
|
|
@ -264,7 +264,7 @@ nav:
|
|||
- 数据科学:
|
||||
- "UCB Data100: Principles and Techniques of Data Science": "数据科学/Data100.md"
|
||||
- 人工智能:
|
||||
- "Neural Networks: Zero to Hero": "人工智能/Neural Networks: Zero to Hero.md"
|
||||
- "Neural Networks: Zero to Hero": "人工智能/Neural Networks:Zero to Hero.md"
|
||||
- "Harvard CS50's Introduction to AI with Python": "人工智能/CS50.md"
|
||||
- "UCB CS188: Introduction to Artificial Intelligence": "人工智能/CS188.md"
|
||||
- 机器学习:
|
||||
|
|
@ -276,9 +276,13 @@ nav:
|
|||
- "CMU 10-414/714: Deep Learning Systems": "机器学习系统/CMU10-414.md"
|
||||
- "MIT6.5940: TinyML and Efficient Deep Learning Computing": "机器学习系统/EML.md"
|
||||
- "Machine Learning Compilation": "机器学习系统/MLC.md"
|
||||
- "UCSD CSE234: Data Systems for Machine Learning": "机器学习系统/CSE234.md"
|
||||
- 深度学习:
|
||||
- "Coursera: Deep Learning": "深度学习/CS230.md"
|
||||
- "国立台湾大学: 李宏毅机器学习": "深度学习/LHY.md"
|
||||
- "CMU 11-785: Introduction to Deep Learning": "深度学习/CMU11-785.md"
|
||||
- "MIT 6.7960: Deep Learning": "深度学习/MIT6-7960.md"
|
||||
- "NYU DLSP21: NYU Deep Learning Spring 2021": "深度学习/NYU-DLSP21.md"
|
||||
- "UMich EECS 498-007 / 598-005: Deep Learning for Computer Vision": "深度学习/EECS498-007.md"
|
||||
- "Stanford CS231n: CNN for Visual Recognition": "深度学习/CS231.md"
|
||||
- "Stanford CS224n: Natural Language Processing": "深度学习/CS224n.md"
|
||||
|
|
@ -286,6 +290,7 @@ nav:
|
|||
- "UCB CS285: Deep Reinforcement Learning": "深度学习/CS285.md"
|
||||
- 深度生成模型:
|
||||
- "学习路线图": "深度生成模型/roadmap.md"
|
||||
- "MIT 6.S184: Generative AI with Stochastic Differential Equations": "深度生成模型/MIT6.S184.md"
|
||||
- "大语言模型":
|
||||
- "CMU 11-868: Large Language Model System": "深度生成模型/大语言模型/CMU11-868.md"
|
||||
- "CMU 11-667: Large Language Models: Methods and Applications": "深度生成模型/大语言模型/CMU11-667.md"
|
||||
|
|
|
|||
Loading…
Reference in a new issue