Skip to content

Commit

Permalink
docs: update README.md
Browse files Browse the repository at this point in the history
Signed-off-by: YdrMaster <[email protected]>
  • Loading branch information
YdrMaster committed Dec 30, 2024
1 parent 4a19e67 commit d55ac10
Show file tree
Hide file tree
Showing 9 changed files with 80 additions and 316 deletions.
75 changes: 11 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,68 +1,15 @@
# 手写 transformer 模型
# InfiniLM

![GitHub Issues or Pull Requests](https://img.shields.io/github/issues/InfiniTensor/transformer-rs)
![GitHub Issues or Pull Requests](https://img.shields.io/github/issues-pr/InfiniTensor/transformer-rs)
![GitHub repo size](https://img.shields.io/github/repo-size/InfiniTensor/transformer-rs)
![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/InfiniTensor/transformer-rs)
![GitHub contributors](https://img.shields.io/github/contributors/InfiniTensor/transformer-rs)
![GitHub commit activity](https://img.shields.io/github/commit-activity/m/InfiniTensor/transformer-rs)
[![CI](https://github.com/InfiniTensor/InfiniLM/actions/workflows/build.yml/badge.svg?branch=main)](https://github.com/InfiniTensor/InfiniLM/actions)
[![license](https://img.shields.io/github/license/InfiniTensor/InfiniLM)](https://mit-license.org/)
[![GitHub Issues](https://img.shields.io/github/issues/InfiniTensor/InfiniLM)](https://github.com/InfiniTensor/InfiniLM/issues)
[![GitHub Pull Requests](https://img.shields.io/github/issues-pr/InfiniTensor/InfiniLM)](https://github.com/InfiniTensor/InfiniLM/pulls)

[**使用指南**](/docs/user-guide/doc.md)
![GitHub repo size](https://img.shields.io/github/repo-size/InfiniTensor/InfiniLM)
![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/InfiniTensor/InfiniLM)
![GitHub contributors](https://img.shields.io/github/contributors/InfiniTensor/InfiniLM)
![GitHub commit activity](https://img.shields.io/github/commit-activity/m/InfiniTensor/InfiniLM)

[YdrMaster/llama2.rs](https://github.com/YdrMaster/llama2.rs) 发展来的手写 transformer 模型项目。

## 使用

> 推荐测试模型:[TinyLlama-1.1B-Chat](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
> 下文所述“模型目录”,需要至少包含下列 3 个文件:
>
> - `config.json`: 模型配置文件;
> - `model.safetesnors`: 模型参数文件;
> - `tokenizer.model`/`vocab.txt`: 分词器词表;
### 转换参数

```plaintext
cargo cast --model <model> --dt <date_type>
```

用于转换参数类型以加速模型加载。

参数:

- `model`: 模型目录;

生成的模型会存放在 `model` 同级目录下,并添加 `_<date_type>` 后缀。

- `date_type`: 参数类型,可为 `f32`/`f16`/`bf16`

### 启动对话服务
[**使用指南**](/docs/user-guide/doc.md) | [**version1**](https://github.com/InfiniTensor/InfiniLM/tree/version1)

```plaintext
cargo chat --model <model>
```

必要参数:

- `model`: 模型目录;

> 目前仅支持 `f16` 精度,必须先转换模型;
其他参数参见 `cargo chat --help`

### 启动文本生成

```plaintext
cargo generate --model <model> --prompt <prompt>
```

必要参数:

- `model`: 模型目录;

> 目前仅支持 `f16` 精度,必须先转换模型。
- `prompt`: 生成文本的开头;

其他参数参见 `cargo generate --help`
[YdrMaster/llama2.rs](https://github.com/YdrMaster/llama2.rs) 发展来的手写 transformer 模型项目。
Binary file removed docs/user-guide/chat-session-error.png
Binary file not shown.
Loading

0 comments on commit d55ac10

Please sign in to comment.