diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml index 30f547366..a9d5ee593 100644 --- a/.github/workflows/docs.yaml +++ b/.github/workflows/docs.yaml @@ -14,7 +14,6 @@ jobs: defaults: run: shell: bash - if: ${{ github.ref == 'refs/heads/main'}} steps: - uses: actions/checkout@v3 diff --git a/blog/2023-07-24-run-llama-2-chat-in-five-minutes.md b/blog/2023-07-24-run-llama-2-chat-in-five-minutes.md new file mode 100644 index 000000000..f3a4e00e4 --- /dev/null +++ b/blog/2023-07-24-run-llama-2-chat-in-five-minutes.md @@ -0,0 +1,90 @@ +--- +title: Run Llama 2-Chat from scratch in 5 minutes +description: docusaurus-plugin-content-blog +slug: run-llama2-chat-in-five-minutes +authors: + - name: Starwhale +tags: [Model Package, llama2] +image: https://starwhale.cn/assets/images/banner-9c279049eb74ef96a8c1eb6ac3636360.jpg +hide_table_of_contents: false +--- + +Meta Llama 2, once released captured the attention of the entire world. [Starwhale](https://starwhale.cn/) has specially developed the Llama 2-Chat and Llama 2-7b model packages. In just 5 minutes, you can engage in a conversation with Llama 2-Chat from scratch on https://cloud.starwhale.cn. + +In the future, we will also provide model packages for Llama 2-13b and Llama 2-70b. Interested friends, please stay tuned! + +The following will provide a detailed introduction to what is Llama 2, what is Starwhale, and how to use Starwhale to run Llama 2-Chat. + + +## What is Llama 2 + +The Llama 2 series models are a set of large language models that utilize optimized autoregressive Transformer architecture. They have undergone pre-training and fine-tuning and come in three parameter versions: 7 billion, 13 billion, and 70 billion. Additionally, Meta has trained a 34 billion parameter version, but it has not been released, and relevant data is mentioned in the research paper. + +Pre-training: Compared to Llama 1, Llama 2's training data has increased by 40%, using 2 trillion tokens for training, and the context length is twice that of Llama 1, reaching 4096. Llama 2 is well-suited for various natural language generation tasks. + +![image](https://github.com/star-whale/docs/assets/101299635/638a0d16-0126-458b-b425-84d9d7d18523) + +Meta compared the results of Llama 2-70b with closed-source models and found that its performance is close to GPT-3.5 on MMLU (Multilingual Multimodal Language Understanding) and GSM8K (German Speech Recognition) tasks. However, there are significant differences in performance on encoding benchmarks. + +Moreover, on almost all benchmarks, Llama 2-70b performs on par with or even better than Google's PaLM-540b model. But there still remains a considerable gap in performance when compared to models like GPT-4 and PaLM-2-L. + +![image](https://github.com/star-whale/docs/assets/101299635/8dd71a5a-471d-412a-8631-c4ada99d8ed2) + +Fine-tuning: Llama 2-Chat is a version of Llama 2 that has been fine-tuned specifically for chat dialogue scenarios. The fine-tuning process involves using SFT (Supervised Fine-Tuning) and RLHF (Reinforcement Learning from Human Feedback) in an iterative optimization to align better with human preferences and improve safety. The fine-tuning data includes publicly available instruction datasets and over one million newly annotated samples. Llama 2-Chat can be used for chat applications similar to virtual assistants. The image below shows the percentage of violations in single-turn and multi-turn conversations. Compared to the baseline, Llama 2-Chat performs particularly well in multi-turn conversations. + +![image](https://github.com/star-whale/docs/assets/101299635/776e31b8-ea32-4a4c-9568-18117f933812) + +## What is Starwhale + +Starwhale is an MLOps platform that offers a comprehensive solution for the entire machine learning operations process. It enables developers and businesses to efficiently and conveniently manage model hosting, execution, evaluation, deployment, and dataset management. Users can choose from three different versions: Standalone, Server, or Cloud, based on their specific requirements. For more detailed information and instructions on using Starwhale, users can refer to the platform's [documentation](https://starwhale.cn/docs/). + +## how to use Starwhale to run Llama 2-Chat + +Workflow：Login → Create a project → Run the model → Chat with Llama2 + +### **1. Login** + +First, you need to log in to the Starwhale platform by clicking on the [login](https://cloud.starwhale.cn/login?lang=zh). If you haven't registered yet, you can click on the [sign-up](https://cloud.starwhale.cn/signup) to create an account. + + +### **2. Create a project** + +After successful login, you will be directed to the project list page. Click on the Create button on the top right corner to create a new project. Enter the project name and click on the Submit button to create the project. + +![image](https://github.com/star-whale/docs/assets/101299635/5228104d-eb26-4504-aa40-838c5bf177c2) + +![image](https://github.com/star-whale/docs/assets/101299635/2494cac8-44f2-4d94-866d-a4d3cc01c453) + +### **3. Run the model** + +Go to the job list page and click on the Create task button. + + 1) Choose the running resource, you can select A100 80G1 (recommended) or A10 24G1. + 2) Select the model: starwhale/public/llama2-7b-chat/ki72ulaf(latest). + 3) Choose the handler: Run the chatbot model, select the default option: evaluation:chatbot. + 4) Choose the runtime: Select the default option, built-in. + 5) Advanced configuration: Turn on the auto-release switch, where you can set the duration after which the task will be automatically canceled. If you don't set auto-release, you can manually cancel the task after the experiment is completed. + +Click on Submit to run the model. + +![image](https://github.com/star-whale/docs/assets/101299635/cc21187a-a40b-44a3-bce7-c785d5fc8d7b) + +#### **4. View the Results and Logs** + +The job list page allows you to view all the tasks in the project. + +![image](https://github.com/star-whale/docs/assets/101299635/5790352b-2d5f-44a5-8ac5-0fe02996b721) + +Click on the Job ID to enter the task details page, and then click on View Logs to see the logs. + +The total time taken from task submission to model execution is 5 minutes. + +![image](https://github.com/star-whale/docs/assets/101299635/c412a427-f5b1-4b34-ab3f-95237f79ced4) + +Once the execution is successful, return to the task list and click on the Terminal button to open the chatbox page. You can now start a conversation with Llama 2-Chat on the chatbox page. + +![image](https://github.com/star-whale/docs/assets/101299635/e6e93ab4-d7ca-4bbb-a89a-14850e36ffcb) + +![image](https://github.com/star-whale/docs/assets/101299635/e75f2221-f7ca-4492-981b-7672a2ed65eb) + +These are the instructions on how to use Starwhale Cloud to run Llama 2-Chat. If you encounter any issues during the process, please feel free to leave a private message. You can also visit the [Starwhale official website](https://starwhale.cn) for more information. Thank you for your attention and support. diff --git a/i18n/zh/docusaurus-plugin-content-blog/2023-07-24-run-llama-2-chat-in-five-minutes.md b/i18n/zh/docusaurus-plugin-content-blog/2023-07-24-run-llama-2-chat-in-five-minutes.md index 69307c562..a9ae4e524 100644 --- a/i18n/zh/docusaurus-plugin-content-blog/2023-07-24-run-llama-2-chat-in-five-minutes.md +++ b/i18n/zh/docusaurus-plugin-content-blog/2023-07-24-run-llama-2-chat-in-five-minutes.md @@ -1,21 +1,21 @@ --- title: 5分钟快速运行Llama 2-Chat -description: Starwhale -slug: run llama2-Chat in five minutes +description: 5分钟快速运行Llama 2-Chat +slug: run-llama2-chat-in-five-minutes authors: - name: Starwhale -tags: [模型] +tags: [模型, llama2] image: https://starwhale.cn/assets/images/banner-9c279049eb74ef96a8c1eb6ac3636360.jpg hide_table_of_contents: false --- -Meta Llama 2 一经发布就吸引了全世界的目光，[Starwhale](https://starwhale.cn/) 特别制作了 Llama 2-Chat 和 Llama 2-7b模型包。只需5分钟，您就可以在https://cloud.starwhale.cn/ 上和 Llama 2-Chat 进行对话。 +Meta Llama 2 一经发布就吸引了全世界的目光，[Starwhale](https://starwhale.cn/) 特别制作了 Llama 2-Chat 和 Llama 2-7b模型包。只需5分钟，您就可以在 上和 Llama 2-Chat 进行对话。 后续我们也将提供Llama 2-13b 和 Llama 2-70b 的模型包，感兴趣的朋友们请持续关注！ 下文将为大家详细介绍什么是 Llama 2，什么是 Starwhale 以及如何使用 Starwhale 运行 Llama 2-Chat。 -## 什么是 Llama 2 +## 什么是 Llama 2 Llama 2 系列模型是一组使用了优化的自回归 Transformer 架构的大语言模型，经过了预训练和微调，包含70亿、130亿和700亿三种参数版本。此外，Meta还训练了 340亿参数版本，但并未发布，相关数据在论文中有体现。 @@ -33,17 +33,17 @@ Meta将 Llama 2-70b 的结果与闭源模型进行了比较，在 MMLU 和 GSM8K ## 什么是 Starwhale -Starwhale是一个MLOps平台，提供MLOps全流程解决方案，能够让开发者和企业高效便捷地进行模型托管、运行、评测、部署及数据集管理等。用户可以根据自己的需要，选择 Standalone、Server 或者 Cloud 任意一版使用，详细说明可参考文档[什么是Starwhale](https://starwhale.cn/docs/) +Starwhale是一个MLOps平台，提供MLOps全流程解决方案，能够让开发者和企业高效便捷地进行模型托管、运行、评测、部署及数据集管理等。用户可以根据自己的需要，选择 Standalone、Server 或者 Cloud 任意一版使用，详细说明可参考文档[什么是Starwhale](https://starwhale.cn/docs/)。 ## 如何使用 Starwhale Cloud 运行 Llama 2-Chat 基本流程：登录账号 → 创建项目 → 运行模型 → 进行对话 -**一. 登录** +### **一. 登录** 首先，需要登录Starwhale平台，点击跳转[登录入口](https://cloud.starwhale.cn/login?lang=zh)。如您尚未注册，可点击 [注册入口](https://cloud.starwhale.cn/signup) 进行注册。 -**二. 创建项目** +### **二. 创建项目** 成功登录后进入项目列表页，点击右上角的**创建**项目按钮，输入项目名称,点击 提交 按钮即可新建一个项目。 @@ -51,12 +51,12 @@ Starwhale是一个MLOps平台，提供MLOps全流程解决方案，能够让开 ![image](https://github.com/star-whale/docs/assets/101299635/2494cac8-44f2-4d94-866d-a4d3cc01c453) -**三. 运行模型** +### **三. 运行模型** 进入作业列表页，点击右上角的**创建**任务按钮。 1) 选择运行资源，可以选择 A100 80G*1（推荐） 或者 A10 24G*1 - 2) 选择模型：starwhale/public/llama2-7b-chat/ki72ulaf(latest) + 2) 选择模型：starwhale/public/llama2-7b-chat/ki72ulaf(latest) 3) 选择handler：运行对话模型，选择默认项：evaluation:chatbot 4) 选择运行时：选择默认项，内置 5) 高级配置，打开自动释放开关：可设置任务自动释放时长，达到设置时长，系统会自动取消任务运行。如不设置自动释放，您可以在体验完成后手动取消任务。 @@ -65,7 +65,7 @@ Starwhale是一个MLOps平台，提供MLOps全流程解决方案，能够让开 ![image](https://github.com/star-whale/docs/assets/101299635/cc21187a-a40b-44a3-bce7-c785d5fc8d7b) -**四. 查看运行结果和日志** +#### **四. 查看运行结果和日志** 作业列表页可以查看项目中的所有作业。 @@ -77,10 +77,10 @@ Starwhale是一个MLOps平台，提供MLOps全流程解决方案，能够让开 ![image](https://github.com/star-whale/docs/assets/101299635/c412a427-f5b1-4b34-ab3f-95237f79ced4) -运行成功后返回任务列表，点击**终端**按钮，可打开 chatbox 页面，在chatbox 页面和 Llama 2-Chat 对话 +运行成功后返回任务列表，点击**终端**按钮，可打开 chatbox 页面，在 chatbox 页面和 Llama 2-Chat 对话 ![image](https://github.com/star-whale/docs/assets/101299635/e6e93ab4-d7ca-4bbb-a89a-14850e36ffcb) ![image](https://github.com/star-whale/docs/assets/101299635/e75f2221-f7ca-4492-981b-7672a2ed65eb) -以上就是关于如何使用 Starwhale Cloud 运行 Llama 2-Chat 的说明，如果您在使用过程中有任何问题欢迎私信留言。您也可以通过[Starwhale官网](https://cloud.starwhale.cn/)了解更多信息，感谢您的关注和支持。 +以上就是关于如何使用 Starwhale Cloud 运行 Llama 2-Chat 的说明，如果您在使用过程中有任何问题欢迎私信留言。您也可以通过[Starwhale官网](https://starwhale.cn/)了解更多信息，感谢您的关注和支持。