add JittorLLMs examples #173
Conversation
README.md
Outdated
| @@ -483,6 +483,10 @@ print(text) | |||
|
|
|||
| 详细的推理文档见：[inference-speed/GPU/vllm_example](https://github.com/FlagAlpha/Llama2-Chinese/blob/main/inference-speed/GPU/vllm_example/README.md) | |||
|
|
|||
| ### JittorLLMs | |||
| [JittorLLMs](https://github.com/Jittor/JittorLLMs)由非十科技领衔，与清华大学可视媒体研究中心合作研发，通过动态swap机制大幅降低硬件配置要求（减少80%）,并且Jittor框架通过零拷贝技术，大模型加载开销降低40%，同时，通过元算子自动编译优化，计算性能提升20%以上。 | |||
There was a problem hiding this comment.
这里的计算开销40%，性能提升20%可以写一下是和哪个框架对比不
|
|
||
| ## 配置要求 | ||
|
|
||
| * 内存要求：至少2G，推荐32G |
There was a problem hiding this comment.
这里配置要求是针对所有的模型吗，还是什么样子的模型呢
|
|
||
| 部署只需一行命令即可： | ||
|
|
||
| ``` |
There was a problem hiding this comment.
这里可以以本地的Atom-7B模型为例子吗 https://huggingface.co/FlagAlpha/Atom-7B
There was a problem hiding this comment.
Atom-7Bd的prompt格式和了lama2不太一样
https://github.com/FlagAlpha/Llama2-Chinese/blob/main/examples/chat_gradio.py#L33
| JittorLLM通过gradio库，允许用户在浏览器之中和大模型直接进行对话。 | ||
|
|
||
| ~~~bash | ||
| python web_demo.py llama2 |
There was a problem hiding this comment.
同样这里可以以本地的Atom-7B模型为例子吗 https://huggingface.co/FlagAlpha/Atom-7B
There was a problem hiding this comment.
Atom-7Bd的prompt格式和了lama2不太一样
https://github.com/FlagAlpha/Llama2-Chinese/blob/main/examples/chat_gradio.py#L33
| JittorLLM在api.py文件之中，提供了一个架设后端服务的示例。 | ||
|
|
||
| ~~~bash | ||
| python api.py llama2 |
There was a problem hiding this comment.
同样这里可以以本地的Atom-7B模型为例子吗 https://huggingface.co/FlagAlpha/Atom-7B
Atom-7Bd的prompt格式和了lama2不太一样
https://github.com/FlagAlpha/Llama2-Chinese/blob/main/examples/chat_gradio.py#L33
No description provided.