Skip to content

Commit

Permalink
update starwhale intro and quickstart docs (#51)
Browse files Browse the repository at this point in the history
  • Loading branch information
tianweidut authored Dec 5, 2023
1 parent 52176e1 commit 014562a
Show file tree
Hide file tree
Showing 10 changed files with 140 additions and 72 deletions.
18 changes: 13 additions & 5 deletions docs/getting-started/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,21 @@
title: Getting started
---

First, you need to install the [Starwhale Client (swcli)](../swcli), which can be done by running the following command:
Each deployment of Starwhale is called an instance. All instances can be managed by the Starwhale Client (swcli).

```bash
python3 -m pip install starwhale
```
You can start using Starwhale with one of the following instance types:

For more information, see the [swcli installation guide](../swcli/installation).
* **Starwhale Standalone** - Rather than a running service, Starwhale Standalone is actually a repository that resides in your local file system. It is created and managed by the Starwhale Client (swcli). You only need to install swcli to use it. Currently, each user on a single machine can have only ONE Starwhale Standalone instance. We recommend you use the Starwhale Standalone to build and test your datasets, runtime, and models before pushing them to Starwhale Server/Cloud instances.
* **Starwhale Server** - Starwhale Server is a service deployed on your local server. Besides text-only results from the Starwhale Client (swcli), Starwhale Server provides Web UI for you to manage your datasets and models, evaluate your models in your local Kubernetes cluster, and review the evaluation results.
* **Starwhale Cloud** - Starwhale Cloud is a managed service hosted on public clouds. By registering an account on <https://cloud.starwhale.cn>, you are ready to use Starwhale without needing to install, operate, and maintain your own instances. Starwhale Cloud also provides public resources for you to download, like datasets, runtimes, and models. Check the "starwhale/public" project on Starwhale Cloud for more details.

When choosing which instance type to use, consider the following:

| Instance Type | Deployment location | Maintained by | User Interface | Scalability |
| ------------- | ------------- | ------------- | ------------- | ------------- |
| Starwhale Standalone | Your laptop or any server in your data center | Not required | Command line | Not scalable |
| Starwhale Server | Your data center | Yourself | Web UI and command line | Scalable, depends on your Kubernetes cluster |
| Starwhale Cloud | Public cloud, like AWS or Aliyun | the Starwhale Team |Web UI and command line | Scalable, but currently limited by the freely available resource on the cloud |

Depending on your instance type, there are three getting-started guides available for you:

Expand Down
8 changes: 6 additions & 2 deletions docs/getting-started/server.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,13 @@
title: Getting started with Starwhale Server
---

## Install Starwhale Server
## Start Starwhale Server

To install Starwhale Server, see the [installation guide](../server/installation/index.md).
```bash
swcli server start
```

For detailed informatiuon, see the [installation guide](../server/installation/index.md).

## Create your first project

Expand Down
8 changes: 8 additions & 0 deletions docs/getting-started/standalone.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@ When the [Starwhale Client (swcli)](../swcli/) is installed, you are ready to us

We also provide a Jupyter Notebook example, you can try it in [Google Colab](https://colab.research.google.com/github/star-whale/starwhale/blob/main/example/notebooks/quickstart-standalone.ipynb) or in your local [vscode/jupyterlab](https://github.com/star-whale/starwhale/blob/main/example/notebooks/quickstart-standalone.ipynb).

## Installing Starwhale Client

```bash
python3 -m pip install starwhale
```

For detailed information, see [Starwhale Client Installation Guide](swcli/installation).

## Downloading Examples

Download Starwhale examples by cloning the Starwhale project via:
Expand Down
6 changes: 6 additions & 0 deletions docs/swcli/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ We can use `swcli` to complete all tasks for Starwhale Instances. `swcli` is wri
DO NOT install Starwhale in your system's global Python environment. It will cause a python dependency conflict problem.
:::

## Quick install

```bash
python3 -m pip install starwhale
```

## Prerequisites

* Python 3.7 ~ 3.11
Expand Down
66 changes: 38 additions & 28 deletions docs/what-is-starwhale.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,31 +5,41 @@ title: What is Starwhale

## Overview

Starwhale is an MLOps/LLMOps platform that make your model creation, evaluation and publication much easier. It aims to create a handy tool for data scientists and machine learning engineers.

Starwhale helps you:

* Keep track of your training/testing dataset history including data items and their labels, so that you can easily access them.
* Manage your model packages that you can share across your team.
* Run your models in different environments, either on a Nvidia GPU server or on an embedded device like Cherry Pi.
* Create a online service with interactive Web UI for your models.

Starwhale is designed to be an open platform. You can create your own plugins to meet your requirements.

## Deployment options

Each deployment of Starwhale is called an instance. All instances can be managed by the Starwhale Client (swcli).

You can start using Starwhale with one of the following instance types:

* **Starwhale Standalone** - Rather than a running service, Starwhale Standalone is actually a repository that resides in your local file system. It is created and managed by the Starwhale Client (swcli). You only need to install swcli to use it. Currently, each user on a single machine can have only ONE Starwhale Standalone instance. We recommend you use the Starwhale Standalone to build and test your datasets, runtime, and models before pushing them to Starwhale Server/Cloud instances.
* **Starwhale Server** - Starwhale Server is a service deployed on your local server. Besides text-only results from the Starwhale Client (swcli), Starwhale Server provides Web UI for you to manage your datasets and models, evaluate your models in your local Kubernetes cluster, and review the evaluation results.
* **Starwhale Cloud** - Starwhale Cloud is a managed service hosted on public clouds. By registering an account on <https://cloud.starwhale.cn>, you are ready to use Starwhale without needing to install, operate, and maintain your own instances. Starwhale Cloud also provides public resources for you to download, like datasets, runtimes, and models. Check the "starwhale/public" project on Starwhale Cloud for more details.

When choosing which instance type to use, consider the following:

| Instance Type | Deployment location | Maintained by | User Interface | Scalability |
| ------------- | ------------- | ------------- | ------------- | ------------- |
| Starwhale Standalone | Your laptop or any server in your data center | Not required | Command line | Not scalable |
| Starwhale Server | Your data center | Yourself | Web UI and command line | Scalable, depends on your Kubernetes cluster |
| Starwhale Cloud | Public cloud, like AWS or Aliyun | the Starwhale Team |Web UI and command line | Scalable, but currently limited by the freely available resource on the cloud |
Starwhale is an MLOps/LLMOps platform that provides R&D operation management capabilities for machine learning projects, establishing standardized model development, testing, deployment and operation processes, connecting business teams, AI teams and operation teams. It solves problems such as long model iteration cycles, team collaboration, and waste of human resources in the machine learning process. Starwhale provides Standalone, Server and Cloud in three instance ways to meet the development needs of a single machine environment, private cluster deployment and multi-cloud services hosted by the Starwhale team.

Starwhale is also an [open source platform](https://github.com/star-whale/starwhale), using the [Apache-2.0 license](https://github.com/star-whale/starwhale/blob/main/LICENSE).

![products](https://starwhale-examples.oss-cn-beijing.aliyuncs.com/docs/products.png)

* Fundamentals:
* [Starwhale Model](model/index): Starwhale Model is a standard package format for machine learning models, which can be used for various purposes, such as model fine-tuning, model evaluation, and online services. Starwhale Model includes model files, inference code, configuration files, etc.
* [Starwhale Dataset](dataset/index): Starwhale Dataset enables efficient data storage, data loading, and data visualization, making it a data management tool for the ML/DL field.
* [Starwhale Runtime](runtime/index): Starwhale Runtime provides a reproducible and shareable runtime environment for running Python programs. With Starwhale Runtime, you can easily share with others and use it on Starwhale Server and Starwhale Cloud instances.
* Model Evaluation:
* [Model Evaluation](evaluation/index): Starwhale Model Evaluation allows users to implement complex, production-level, distributed model evaluation tasks with minimal Python code using the SDK.
* Live Demo: Evaluate models online through a Web UI.
* Reports: Create shareable, automatically integrated evaluation reports.
* Tables: Provide multi-dimensional model evaluation result comparisons and displays, with support for multimedia data such as images, audio, and video. The tables can present all the data and artifacts recorded during the evaluation process using the Starwhale Python SDK.
* LLM Fine-tuning: Provide a full toolchain for LLM fine-tuning, including model fine-tuning, batch evaluation comparison, online evaluation comparison, and model publishing.
* Deployment Instances:
* Starwhale Standalone: Deployed in a local development environment, managed by the `swcli` command-line tool, meeting development and debugging needs.
* Starwhale Server: Deployed in a private data center, relying on a Kubernetes cluster, providing centralized, web-based, and secure services.
* Starwhale Cloud: Hosted on a public cloud, with the access address <https://cloud.starwhale.ai>. The Starwhale team is responsible for maintenance, and no installation is required. You can start using it after registering an account.

## Typical Use Cases

* **Dataset Management**: With the Starwhale Dataset Python SDK, you can easily import, create, distribute, and load datasets while achieving fine-grained version control and visualization.
* **Model Management**: By using a simple packaging mechanism, you can generate Starwhale Model packages that include models, configuration files, and code, providing efficient distribution, version management, Model Registry, and visualization, making the daily management of model packages more straightforward.
* **Machine Learning Runtime Sharing**: By exporting the development environment or writing a simple YAML, you can reproduce the environment in other instances, achieving a stable and consistent runtime. Starwhale Runtime abstracts and shields some underlying dependencies, so users don't need to master Dockerfile writing or CUDA installation, making it easy to define an environment that meets the requirements of machine learning programs.
* **Model Evaluation**: With the Starwhale Evaluation Python SDK, you can implement efficient, large-scale, multi-dataset, and multi-stage model evaluations in a distributed cluster environment with minimal code, record data and artifacts generated during the evaluation process in Starwhale Tables, and provide various visualization methods.
* **Online Evaluation**: Quickly create interactive Web UI online services for Starwhale models to perform rapid testing.
* **Model Fine-tuning**: Provide a complete toolchain for fine-tuning large language models (LLMs), making the model fine-tuning process faster and more quantifiable.

Starwhale is an open platform that can be used for individual functions or combined for use, with the core goal of providing a convenient tool for data scientists and machine learning engineers to improve work efficiency.

## Start Your Starwhale Journey

* Complete the [installation of Starwhale Client](swcli/installation) and [launch of Starwhale Server](server/installation/server-start) within 5-10 minutes.
* Follow the [Starwhale Standalone Getting Started Guide](server/installation/server-start) to build, evaluate, and visualize the helloworld example's model, dataset, and runtime in your local environment.
* Refer to the [Starwhale Server Getting Started Guide](getting-started/server) to run the helloworld example's model evaluation in Starwhale Server.
* Read the [User Guide](swcli) and [Reference Guide](reference/swcli), and refer to the [examples](https://github.com/starwhale-ai/starwhale/tree/main/examples) to explore more features and functionalities.
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,19 @@
title: 入门指南
---

首先,您需要安装[Starwhale Client(swcli)](../swcli),可以运行如下命令:
Starwhale的每个部署称为一个实例。所有实例都可以通过Starwhale Client(swcli)进行管理。您可以任选以下实例类型之一开始使用:

```bash
python3 -m pip install starwhale
```
* **Starwhale Standalone** - Starwhale Standalone 本质上是一套存储在本地文件系统中的数据库。它由 Starwhale Client(swcli)创建和管理。您只需安装 `swcli` 即可使用。目前,一台机器上的每个用户只能拥有一个Starwhale Standalone 实例。我们建议您使用 Starwhale Standalone 来构建和测试您的数据集和模型,然后再将它们推送到 Starwhale Server/Cloud 实例。
* **Starwhale Server** - Starwhale Server 是部署在您本地服务器上的服务。除了 Starwhale Client(swcli)的文本交互界面,Starwhale Server还提供 Web UI供您管理数据集和模型,以及在Kubernetes集群中运行模型并查看运行结果。
* **Starwhale Cloud** - Starwhale Cloud 是托管在公共云上的服务。 通过在<https://cloud.starwhale.cn>注册一个账号,您就可以使用Starwhale,而无需安装、运行和维护您自己的实例。 Starwhale Cloud 还提供公共资源供您下载,例如一些流行的开源集数据集、模型和运行时。查看 Starwhale Cloud 实例上的 “starwhale/public”项目以获取更多详细信息。

更多详细信息请参阅[swcli安装指南](../swcli/installation)
在您决定要使用的实例类型时,请考虑以下因素:

| 实例类型 | 部署位置 | 维护者 | 用户界面 | 可扩展性 |
| -------------- | -------------- | -------------- | -------------- | -------------- |
| Starwhale Standalone | 您的笔记本电脑或本地服务器 | 不需要 | 命令行 | 不可扩展 |
| Starwhale Server | 您的数据中心 | 您自己 | Web UI和命令行 | 可扩展,取决于您的 Kubernetes 集群 |
| Starwhale Cloud | 公共云,如AWS或阿里云 | Starwhale团队 | Web UI和命令行 | 可扩展,但目前受到云上免费可用资源的限制 |

根据您使用的实例类型,您可以参考以下三个入门指南:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,13 @@
title: Starwhale Server入门指南
---

## 安装Starwhale Server
## 启动 Starwhale Server

安装 Starwhale Server,参见[安装指南](../server/installation/index.md)
```bash
swcli server start
```

更多的Starwhale Server 安装和启动信息,请参见[安装指南](../server/installation/index.md)

## 创建您的第一个项目

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,16 @@
title: Starwhale Standalone入门指南
---

[Starwhale Client(swcli)](../swcli/)安装完成后,您就可以使用Starwhale Standalone。

我们也提供对应的Jupyter Notebook例子,可以在 [Google Colab](https://colab.research.google.com/github/star-whale/starwhale/blob/main/example/notebooks/quickstart-standalone.ipynb) 或本地的 [vscode/jupyterlab](https://github.com/star-whale/starwhale/blob/main/example/notebooks/quickstart-standalone.ipynb) 中试用。

## 安装 Starwhale Client

```bash
python3 -m pip install starwhale
```

详细文档参见 [Starwhale Client 安装指南](swcli/installation)

## 下载例子

通过以下方式克隆Starwhale项目来下载Starwhale示例:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ title: 安装指南
非常不建议将 Starwhale 安装在系统的全局 Python 环境中,可能会导致 Python 的依赖冲突问题。使用 venv 或 conda 创建一个隔离的 Python 环境,并在其中安装 Starwhale,是 Python 推荐的做法。
:::

## 快速安装

```bash
python3 -m pip install starwhale
```

## 先决条件

* Python3.7 ~ 3.11
Expand Down
Loading

0 comments on commit 014562a

Please sign in to comment.