update large language model

This commit is contained in:
Yuhao Chen 2024-11-25 22:03:32 -05:00
parent e40fd2851a
commit 17e2b3423b
9 changed files with 59 additions and 47 deletions

View file

@ -0,0 +1,27 @@
# Stanford CS324: Large Language Models
## Course Introduction
- **University**: Stanford University
- **Prerequisites**: None specified, but familiarity with natural language processing and machine learning is recommended.
- **Course Difficulty**: 🌟🌟🌟🌟🌟
- **Course Website**: [CS324: Large Language Models](https://stanford-cs324.github.io/winter2022/)
"CS324: Large Language Models" is a graduate-level course at Stanford University that explores the fundamentals, theory, ethics, and system aspects of large language models (LLMs). The course also offers hands-on experience in evaluating and building these models.
Massive pre-trained LLMs have revolutionized the field of natural language processing (NLP), enabling state-of-the-art performance across numerous tasks and demonstrating the ability to generate fluent text and perform few-shot learning. However, these models are challenging to interpret and introduce new ethical and scalability concerns. This course provides students with a comprehensive understanding of LLMs and practical exposure through projects and paper discussions.
### Coursework
1. **Paper Reviews and Discussions** (20%)
- Write reviews for assigned papers.
- Participate in at least two student panels to lead discussions.
2. **Projects** (2 × 40% = 80%)
- **Project 1**: Evaluate the capabilities and risks of language models (e.g., GPT-3).
- **Project 2**: Build and improve language models using tools like BERT-base.
---

View file

@ -0,0 +1,28 @@
# Stanford CS324: Large Language Models
## 课程简介
- **所属大学**: 斯坦福大学
- **先修要求**: 未指定,但建议具备自然语言处理和机器学习的基础知识。
- **课程难度**: 🌟🌟🌟🌟🌟
- **课程网站**: [CS324: 大型语言模型](https://stanford-cs324.github.io/winter2022/)
《CS324: 大型语言模型》是斯坦福大学开设的一门研究生课程探讨大型语言模型LLMs的基础理论、伦理问题、系统架构等方面并提供评估和构建这些模型的实践机会。
大规模预训练的 LLMs 革新了自然语言处理NLP领域实现了多个任务的最先进性能并展现了生成流畅文本和少样本学习的能力。然而这些模型难以理解同时带来了新的伦理和可扩展性挑战。本课程旨在让学生全面理解 LLMs并通过项目和论文讨论获得实践经验。
### 课程作业
1. **论文评审与讨论**20%
- 撰写所分配论文的评审。
- 至少参与两次学生小组讨论并主导讨论。
2. **项目**2 × 40% = 80%
- **项目 1**: 评估语言模型(例如 GPT-3的能力与风险。
- **项目 2**: 使用 BERT-base 等工具构建和改进语言模型。
项目需以 1-2 人小组完成,使用清晰的格式编排,并以 PDF 提交。截止时间为每次晚上 11:00 PST通过 Gradescope 提交。
---

View file

@ -1,24 +0,0 @@
# UC Berkeley CS 194/294-267: Understanding Large Language Models: Foundations and Safety
## Course Introduction
- **University**: UC Berkeley
- **Prerequisites**: CS 182/282A Deep Neural Networks or equivalent, with hands-on deep learning experience.
- **Course Difficulty**: 🌟🌟🌟🌟🌟🌟
- **Course Website**: [Understanding Large Language Models](http://rdi.berkeley.edu/understanding_llms/s24)
- **Lecture Video**<https://www.youtube.com/playlist?list=PLJ66BAXN6D8H_gRQJGjmbnS5qCWoxJNfe>
"Understanding Large Language Models: Foundations and Safety" is a Spring 2024 course at UC Berkeley, co-taught by **Professor Dawn Song** and **Dan Hendrycks**, with **Yu Gai** as the GSI. This course explores the foundational principles, interpretability, scaling laws, and risks associated with large language models (LLMs) such as ChatGPT.
The course provides a rigorous introduction to LLMs, discussing their emergence, limitations, and potential risks, as well as methods for safer and more beneficial applications. Topics covered include:
- Foundations of LLMs
- Interpretability
- Scaling laws
- Adversarial robustness
- AI alignment and governance
- Privacy, watermarking, and Trojans
- Agency, emergence, reasoning, and mathematics
- Evaluation and benchmarking

View file

@ -1,23 +0,0 @@
# UC Berkeley CS 194/294-267 Understanding Large Language Models: Foundations and Safety
## 课程简介
- **所属大学**UC Berkeley
- **先修要求**CS 182/282A 深度神经网络课程或等效课程,并具有深度学习的实际操作经验。
- **课程难度**:🌟🌟🌟🌟🌟🌟
- **课程网站**<http://rdi.berkeley.edu/understanding_llms/s24>
- **课程视频**<https://www.youtube.com/playlist?list=PLJ66BAXN6D8H_gRQJGjmbnS5qCWoxJNfe>
《理解大型语言模型:基础与安全》是 UC Berkeley 在 2024 年春季开设的一门课程,由 **Dawn Song 教授****Dan Hendrycks** 联合授课,助教为 **Yu Gai**。本课程重点探讨大型语言模型(如 ChatGPT的基础原理、可解释性、扩展定律以及相关风险。
课程内容包括:
- 大型语言模型的基础
- 可解释性
- 扩展定律
- 对抗鲁棒性
- 人工智能对齐与治理
- 隐私、水印与木马
- 代理性、涌现性、推理与数学
- 评估与基准测试

View file

@ -280,4 +280,8 @@ nav:
- "Columbia STAT 8201: Deep Generative Models": "机器学习进阶/STAT8201.md"
- "U Toronto STA 4273 Winter 2021: Minimizing Expectations": "机器学习进阶/STA4273.md"
- "Stanford STATS214 / CS229M: Machine Learning Theory": "机器学习进阶/CS229M.md"
- 大型语言模型与生成模型:
- "CMU CS11-667: Large Language Models Methods and Application": "大型语言模型与生成模型/CS11-667.md"
- "Stanford CS324: Large Language Models" : "大型语言模型与生成模型/CS324.md"
- "UCB CS194/294-267: Understanding Large Language Models": "大型语言模型与生成模型/CS194-267.md"
- 后记: "后记.md"