NOTE: AntRay-2.41.0 is align with the community version ray-2.41.0. The following content describes the incremental update features of ant-ray-2.41.0 compared to the community ray-2.41.0
Highlight: Ray Virtual Cluster Developer Preview
This is a new specific feature called Ray Virtual Clusters.
Virtual cluster provides logical resource isolation, enables multi-tenancy by allowing Ray cluster admins to easily manage multiple workloads on the same Ray cluster.
A virtual cluster consists of a group of heterogeneous worker nodes and a collection of child clusters, including divisible cluster and indivisible cluster:
Users can submit jobs for execution on the Ray virtual cluster through the HTTP API or by using command-line tools. See Virtual Cluster Management API and Virtual Cluster CLI for more information.
🎉 Please refer to Ray Virtual Clusters for more details.
Ray Libraries
Ray Data
🎉 New Features:
- Add Ray data virtual cluster test (#453)
Ray Core and Ray Clusters
🎉 New Features:
- [1/N] add virtual cluster grpc service and handler (#410)
- [2/N][VirtualClusetr] add GcsVirtualClusterTable (#411)
- [3/N][VirtualCluster] pass virtual cluster id to gcs when submitting job (#413)
- [4/N][VirtualCluster] Add virtual cluster pub/sub channel (#414) (#415)
- [5/N][VirtualCluster] Pass virtual_cluster_id through job submission and scheduling (#416)
- [6/N][VirtualCluster] Implement CreateOrUpdateVirtualCluster (#420)
- [7/N][VirtualCluster] remove virtual cluster name (#421)
- [9/N][VirtualCluster] Implement RemoveLogicalCluster (#423)
- [10/N][VirtualCluster] Implement GetVirtualClusters (#424)
- [11/N][VirtualCluster] Add VirtualClusterInfoAccessor (#427)
- [12/N][VirtualCluster] Add Virtual Cluster Support for Job, Actor, and Task Scheduling (#428)
- [VirtualCluster] Dashboard supports CRUD of virtual clusters [1/N] (#418)
- [13/N][VirtualCluster] Add VirtualClusterManager to raylet (#429)
- [14/N][VirtualCluster] Raylet only subscribe mixed cluster data (#431)
- [15/N][VirtualCluster] Complete scheduling for mixed virtual cluster (#433)
- [16/N][VirtualCluster] Add virtual cluster failover logic (#434)
- [17/N][VirtualCluster] Support exclusive virtual cluster in Job Submission (#435)
- [18/N][VirtualCluster] Handle job finished event in gcs virtual cluster manager (#436)
- [19/N][VirtualCluster] Improve the virtual cluster failover logic (#437)
- [20/N][VirtualCluster] Fix virtual cluster scheduling for Job Submission (#439)
- refactor mixed/exclusive cluster to indivisible/divisible cluster (#443)
- [22/N][VirtualCluster] Implement Resubscribe for virtual cluster info accessor (#444)
- [23/N][VirtualCluster]add node_instances_map to speed up the lookup of node instances (#445)
- [24/N][VirtualCluster] Handle detached job cluster (#446)
- [25/N][VirtualCluster] Introduce undevided node instances concept (#447)
- [VirtualCluster] ray.nodes fetch only the virtual cluster nodes (#442)
- [VirtualCluster] fix code format (#449)
- [26/N][VirtualCluster] Add ray list and get vclusters api (#450)
- [27/N][VirtualCluster] Remove Job Cluster asynchronously (#451)
- [VirtualCluster] Clean up after removing node instances from a virtual cluster (#441)
- lint (#456)
- Fix (#454)
- add bazel test target (#458)
- [VirtualCluster] Delay local node cleanup function (#459)
- Enhance stability and reduce execution time of virtual cluster unittest (#460)
- [29/N][VirtualCluster]Add virtual cluster to job submit cli command (#461)
- [VirtualCluster] Better UT (#462)
- [VirtualCluster] Fix Listener error log for detached actor/pg listeners (#463)
- [VirtualCluster] Add create/remove options to cli (#464)
- [VirtualCluster] Check virtual cluster name duplication when creating vclusters (#465)
- [VirtualCluster] vcluster cli command support minimal install (#467)
- [VirtualCluster] enrich vcluster list command (#468)
- [VirtualCluster] Prevent abnormal removal of VirtualCluster instances (#469)
🔨 Fixes:
- [Fix][Core] Fix compile error about IndexOf (#408)
📖 Documentation:
- add virtual cluster documents (#470)
- Create deploy-docs (#473)
- Fix deploy-doc (#474)
- Add document deploy yaml (#475)
- update docment base url (#477)
Thanks
@wumuzi520, @xsuler, @NKcqx, @Chong-Li, @mingcheng, @Qstar
Huge thanks to all our contributors for making this release possible.