Skip to content

@ModelCloud ModelCloud.ai

Change the repository type filter

All

    Repositories list

    3 repositories

    • GPTQModel

      Public
      Production ready LLM model compression/quantization toolkit with accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.
      Python
      •
      Apache License 2.0
      •40•264•7•6•Updated Feb 8, 2025Feb 8, 2025

    • Tokenicer

      Public
      Python
      •
      Apache License 2.0
      •1•0•0•1•Updated Feb 8, 2025Feb 8, 2025

    • Device-SMI

      Public
      Self-contained Python lib with zero-dependencies that give you a unified device properties for gpu, cpu, and npu. No more calling separate tools such as nvidia-smi or /proc/cpuinfo and parsing it yourself.
      devicecpugpusminpuxpu
      Python
      •
      Apache License 2.0
      •1•10•1•2•Updated Jan 10, 2025Jan 10, 2025