SGLang is a fast serving framework for large language models and vision language models.

A guidance language for controlling large language models.

The LLM Evaluation Framework

JupyterLab computational environment.

Manage GPU clusters for running AI models

ModelScope: bring the notion of Model-as-a-Service to life.

Heterogeneous AI Computing Virtualization Middleware

Use PEFT or Full-parameter to finetune 400+ LLMs (Qwen2.5, InternLM3, GLM4, Llama3.3, Mistral, Yi1.5, Baichuan2, DeepSeek3, ...) and 150+ MLLMs (Qwen2-VL, Qwen2-Audio, Llama3.2-Vision, Llava, Inter…

A high-throughput and memory-efficient inference and serving engine for LLMs

This repository contains tutorials and examples for Triton Inference Server

📖A curated list of Awesome LLM/VLM Inference Papers with codes, such as FlashAttention, PagedAttention, Parallelism, etc. 🎉🎉

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

AISystem 主要是指AI系统，包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术

HAMi-core compiles libvgpu.so, which ensures hard limit on GPU in container

Ingress NGINX Controller for Kubernetes

sidecar for open kruise

Kubernetes performance and scale test orchestration framework written in golang

Analyzes resource usage and performance characteristics of running containers.

CAT 作为服务端项目基础组件，提供了 Java, C/C++, Node.js, Python, Go 等多语言客户端，已经在美团点评的基础架构中间件框架（MVC框架，RPC框架，数据库框架，缓存框架等，消息队列，配置系统等）深度集成，为美团点评各业务线提供系统丰富的性能指标、健康状况、实时告警等。

go-wrk - a HTTP benchmarking tool based in spirit on the excellent wrk tool (https://github.com/wg/wrk)

The official Meta Llama 3 GitHub site

Clean up Kubernetes yaml and json output to make it readable

The container platform tailored for Kubernetes multi-cloud, datacenter, and edge management ⎈ 🖥 ☁️

an unified scheduler for online and offline tasks

Hubble - Network, Service & Security Observability for Kubernetes using eBPF

A constant throughput, correct latency recording variant of wrk