-
Notifications
You must be signed in to change notification settings - Fork 25
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: YdrMaster <[email protected]>
- Loading branch information
Showing
9 changed files
with
80 additions
and
316 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,68 +1,15 @@ | ||
# 手写 transformer 模型 | ||
# InfiniLM | ||
|
||
![GitHub Issues or Pull Requests](https://img.shields.io/github/issues/InfiniTensor/transformer-rs) | ||
![GitHub Issues or Pull Requests](https://img.shields.io/github/issues-pr/InfiniTensor/transformer-rs) | ||
![GitHub repo size](https://img.shields.io/github/repo-size/InfiniTensor/transformer-rs) | ||
![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/InfiniTensor/transformer-rs) | ||
![GitHub contributors](https://img.shields.io/github/contributors/InfiniTensor/transformer-rs) | ||
![GitHub commit activity](https://img.shields.io/github/commit-activity/m/InfiniTensor/transformer-rs) | ||
[![CI](https://github.com/InfiniTensor/InfiniLM/actions/workflows/build.yml/badge.svg?branch=main)](https://github.com/InfiniTensor/InfiniLM/actions) | ||
[![license](https://img.shields.io/github/license/InfiniTensor/InfiniLM)](https://mit-license.org/) | ||
[![GitHub Issues](https://img.shields.io/github/issues/InfiniTensor/InfiniLM)](https://github.com/InfiniTensor/InfiniLM/issues) | ||
[![GitHub Pull Requests](https://img.shields.io/github/issues-pr/InfiniTensor/InfiniLM)](https://github.com/InfiniTensor/InfiniLM/pulls) | ||
|
||
[**使用指南**](/docs/user-guide/doc.md) | ||
![GitHub repo size](https://img.shields.io/github/repo-size/InfiniTensor/InfiniLM) | ||
![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/InfiniTensor/InfiniLM) | ||
![GitHub contributors](https://img.shields.io/github/contributors/InfiniTensor/InfiniLM) | ||
![GitHub commit activity](https://img.shields.io/github/commit-activity/m/InfiniTensor/InfiniLM) | ||
|
||
从 [YdrMaster/llama2.rs](https://github.com/YdrMaster/llama2.rs) 发展来的手写 transformer 模型项目。 | ||
|
||
## 使用 | ||
|
||
> 推荐测试模型:[TinyLlama-1.1B-Chat](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)。 | ||
> 下文所述“模型目录”,需要至少包含下列 3 个文件: | ||
> | ||
> - `config.json`: 模型配置文件; | ||
> - `model.safetesnors`: 模型参数文件; | ||
> - `tokenizer.model`/`vocab.txt`: 分词器词表; | ||
### 转换参数 | ||
|
||
```plaintext | ||
cargo cast --model <model> --dt <date_type> | ||
``` | ||
|
||
用于转换参数类型以加速模型加载。 | ||
|
||
参数: | ||
|
||
- `model`: 模型目录; | ||
|
||
生成的模型会存放在 `model` 同级目录下,并添加 `_<date_type>` 后缀。 | ||
|
||
- `date_type`: 参数类型,可为 `f32`/`f16`/`bf16`; | ||
|
||
### 启动对话服务 | ||
[**使用指南**](/docs/user-guide/doc.md) | [**version1**](https://github.com/InfiniTensor/InfiniLM/tree/version1) | ||
|
||
```plaintext | ||
cargo chat --model <model> | ||
``` | ||
|
||
必要参数: | ||
|
||
- `model`: 模型目录; | ||
|
||
> 目前仅支持 `f16` 精度,必须先转换模型; | ||
其他参数参见 `cargo chat --help`。 | ||
|
||
### 启动文本生成 | ||
|
||
```plaintext | ||
cargo generate --model <model> --prompt <prompt> | ||
``` | ||
|
||
必要参数: | ||
|
||
- `model`: 模型目录; | ||
|
||
> 目前仅支持 `f16` 精度,必须先转换模型。 | ||
- `prompt`: 生成文本的开头; | ||
|
||
其他参数参见 `cargo generate --help`。 | ||
从 [YdrMaster/llama2.rs](https://github.com/YdrMaster/llama2.rs) 发展来的手写 transformer 模型项目。 |
Binary file not shown.
Oops, something went wrong.