Skip to content

Commit

Permalink
add starwhale framework (#52)
Browse files Browse the repository at this point in the history
  • Loading branch information
tianweidut authored Dec 6, 2023
1 parent 014562a commit b42ba89
Show file tree
Hide file tree
Showing 2 changed files with 44 additions and 2 deletions.
23 changes: 22 additions & 1 deletion docs/what-is-starwhale.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ Starwhale is an MLOps/LLMOps platform that provides R&D operation management cap

Starwhale is also an [open source platform](https://github.com/star-whale/starwhale), using the [Apache-2.0 license](https://github.com/star-whale/starwhale/blob/main/LICENSE).

## Starwhale Products

![products](https://starwhale-examples.oss-cn-beijing.aliyuncs.com/docs/products.png)

* Fundamentals:
Expand All @@ -26,7 +28,26 @@ Starwhale is also an [open source platform](https://github.com/star-whale/starwh
* Starwhale Server: Deployed in a private data center, relying on a Kubernetes cluster, providing centralized, web-based, and secure services.
* Starwhale Cloud: Hosted on a public cloud, with the access address <https://cloud.starwhale.ai>. The Starwhale team is responsible for maintenance, and no installation is required. You can start using it after registering an account.

## Typical Use Cases
## Starwhale Framework

<img src="https://starwhale-examples.oss-cn-beijing.aliyuncs.com/docs/framework.png" alt="framework" width="75%"/>

* For Individual Developers
* Use Starwhale Standalone instances.
* `swcli`: A set of command-line tools written in Python that can be used to complete tasks such as model building, evaluation, and copying.
* Starwhale Python SDK: A series of utility functions or classes that simplify tasks such as model evaluation and fine-tuning, dataset construction, and loading.
* For Team Users
* Use Starwhale Server or Starwhale Cloud instances, which provide a Web UI.
* Core Components:
* **api server**: Depends on MySQL database and object storage, and implements the API for the Starwhale platform.
* **scheduler**: Depends on Docker or Kubernetes, and schedules evaluation tasks.
* **datastore**: Depends on object storage, providing a Big Table-like storage and access method to meet the storage and retrieval requirements of datasets and evaluation data.
* External Dependencies:
* Storage: MySQL and object storage.
* Containers: Docker (single-node) or Kubernetes (cluster).
* Compute Resources: Bare metal or public cloud.

## Starwhale Use Cases

* **Dataset Management**: With the Starwhale Dataset Python SDK, you can easily import, create, distribute, and load datasets while achieving fine-grained version control and visualization.
* **Model Management**: By using a simple packaging mechanism, you can generate Starwhale Model packages that include models, configuration files, and code, providing efficient distribution, version management, Model Registry, and visualization, making the daily management of model packages more straightforward.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ Starwhale是一个 MLOps/LLMOps平台,面向机器学习项目提供研发运

Starwhale 同时也是一个[开源的平台](https://github.com/star-whale/starwhale),使用 [Apache-2.0 协议](https://github.com/star-whale/starwhale/blob/main/LICENSE)

## Starwhale 功能

![products](https://starwhale-examples.oss-cn-beijing.aliyuncs.com/docs/products.png)

* 平台基础:
Expand All @@ -26,6 +28,25 @@ Starwhale 同时也是一个[开源的平台](https://github.com/star-whale/star
* Starwhale Server:部署在私有数据中心里,依赖 Kubernetes 集群,提供集中化的、Web交互式的、安全的服务。
* Starwhale Cloud:托管在公共云上的服务,访问地址为<https://cloud.starwhale.cn>,由 Starwhale 团队负责运维,无需安装,注册账户后即可使用。

## Starwhale 框架

<img src="https://starwhale-examples.oss-cn-beijing.aliyuncs.com/docs/framework.png" alt="framework" width="75%"/>

* 面向独立开发者
* 使用 Starwhale Standalone 实例。
* `swcli`:Python 编写的一组命令行工具,可以用来完成模型构建、评测、拷贝等工作。
* Starwhale Python SDK:一系列工具函数或类,让模型评测与微调、数据集构建与加载等任务变得简单。
* 面向团队用户
* 使用 Starwhale Server 或 Starwhale Cloud 实例,提供Web UI。
* 核心组件:
* **api server**:依赖MySQL数据库和对象存储,完成Starwhale平台的API实现。
* **scheduler**:依赖Docker或Kubernetes,实现评测任务的调度。
* **datastore**:依赖对象存储,提供类似Big Table的存储与访问方式,满足数据集和评测数据的存储、检索等需求。
* 外部依赖:
* 存储:MySQL和对象存储。
* 容器:Docker(单机)或Kubernetes(集群)。
* 计算资源:裸金属或公有云。

## 典型使用场景

* **数据集管理**:基于 Starwhale Dataset Python SDK 可以非常容易的导入、创建、分发和加载数据集,同时可以实现数据集细粒度的版本控制和可视化等功能。
Expand All @@ -40,6 +61,6 @@ Starwhale 是一个开放的平台,工作中可以只用某些功能,也可
## 开始 Starwhale 之旅

* 5-10分钟内完成 [Starwhale Client 的安装](swcli/installation)[Starwhale Server 的启动](server/installation/server-start)
* 参考 [Starwhale Standalone 入门指南](server/installation/server-start),在本地对 helloworld 例子的模型、数据集和运行时构建,完成对 MNIST 数据集的模型效果的评估。
* 参考 [Starwhale Standalone 入门指南](server/installation/server-start),在本地对 helloworld 例子的模型、数据集和运行时进行构建,完成对 MNIST 数据集的模型效果的评估。
* 参考 [Starwhale Server 入门指南](getting-started/server),在 Starwhale Server 中运行 helloworld 例子的模型评测。
* 阅读 [用户指南](swcli)[参考指南](reference/swcli),并参照[例子](https://github.com/star-whale/starwhale/tree/main/example),制作自己的数据集、运行时和模型包,进行模型评测等任务。

0 comments on commit b42ba89

Please sign in to comment.