New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

minimal-llama最小llm #163

Open

ziwang-com opened this issue Jun 25, 2023 · 0 comments

Owner

ziwang-com commented Jun 25, 2023

https://github.com/zphang/minimal-llama
最小法学硕士
此存储库包含用于运行和微调 LLaMA 的随机代码。许多部分仍在进行中。应该有比这里介绍的更有效的调优方法（DeepSpeed / ZeRO，NeoX），但人们可能会发现这已经很有用了。

标记化数据集
 使用 8 位进行 PEFT 微调
 使用朴素管道并行进行微调
（新）使用 8 位和流水线并行进行 PEFT 微调
 杂项说明
这段代码很快就被拼凑在一起，可能包含很多很多错误。欢迎反馈！

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment