-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
4 changed files
with
127 additions
and
54 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| --- | ||
| title: 2025-W02-01 | ||
| tags: | ||
| - "周记" | ||
| categories: dairy | ||
| date: 2025-01-06T18:04:24+08:00 | ||
| modify: 2025-01-06T18:04:24+08:00 | ||
| dir: dairy | ||
| share: false | ||
| cdate: 2025-01-06 | ||
| mdate: " 2025-01-06 " | ||
| --- | ||
|
|
||
| # 2025-W02-01 | ||
|
|
||
| ## Review | ||
|
|
||
| ### Completed | ||
|
|
||
| - [x] 实验室打工 | ||
| - 换了 pipeline 中的一个步骤，效果变好了很多，然后就造了几天的数据。 | ||
| - 感觉一小部分的工作差不多，然后要开始训练了？ | ||
| - [x] 学了点 RL | ||
| - 其实感觉还是挺有意思的，多臂老虎机，MCTS 搜索，然后看着别人的仓库写了个 [alpha zero](https://github.com/WncFht/alpha_zero)。 | ||
| - 去了解了一下 DPO, 看了点视频，也挺有意思的。 | ||
| - [x] 捡起来 CS61C | ||
| - 星期六心情不好，然后 emo 了一整天，晚上想了想，感觉还是要把基础的学了，就回去继续做 CS61C 了。 | ||
| - 还是挺有意思的，看了 4 个 lecture，做了 2 个 lab，把 proj3 做了。 | ||
| - proj3 造了个基于 RISC-V 的 cpu，最后写了个二级流水线，就处理了一下跳转语句的 hazard（还是最 low 的，就搞点 bubble 就好了。本来还以为会有 forwarding 之类的）。但是确实，搭线累的一批，对指令也是熟悉了一点。 | ||
| - 几个 lab 就是主要是为 proj 铺垫的一些用来熟悉 logisim 的，和一些关于 cache 的知识：block_size，TIOl，LRU，associativity 之类的，不得不说 61C 老师是真有激情。中间去听了 15-213 的讲 cache 的，直接就把这些概念扔出来了，感觉不如 61C 讲得好。 | ||
| - [x] 跟着 Andrej Karpathy 写了个 mirograd | ||
| - 大概就是写个 mini pytorch，不过只能支持一些四则运算/指数啥的，主要就是 chain rule 的运用。 | ||
| - 但是讲得真好啊，一层一层展开，他的代码能力也是很强，信手拈来很多实现。感觉对代码实现的理解更细致了一些。 | ||
|
|
||
| ### Uncompleted | ||
|
|
||
| - [ ] 调整作息 | ||
| - 基本每天还是睡的晚，焦虑，迷茫。 | ||
| - [ ] 看论文 | ||
| - 没怎么看，主要是没有方向。最近还是先学学传统 CS 吧。 | ||
|
|
||
| ## Next Week Plan | ||
|
|
||
| - [ ] 也许继续学 CS61C，也许学 15-213, 也许去看 OSTEP 然后开 081 了。没想好。 | ||
| - [ ] 写个简单的 GPT2 ？主要是整理以前学过的知识。 | ||
| - [ ] 继续学 RL，然后读读 RLHF 的工作，看看 DPO 具体实现。 | ||
| - [ ] 实验室打工，估计下个星期工作量不会小。 | ||
| - [ ] 回高中宣讲。 | ||
|
|
||
| ## THOUGHTS | ||
|
|
||
| - emo，感觉那条路都不好走。想到如果要做大模型就要有卡，然后就要去实习。然后 GPA 就不保，学校这里没用的课还好多，还要签到。 | ||
| - 感觉基本一直都是一个人在路上，东搞搞西搞搞。🤣，大概我太封闭了。 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters