Skip to content

Latest commit

 

History

History
38 lines (28 loc) · 1.77 KB

environment.md

File metadata and controls

38 lines (28 loc) · 1.77 KB

Prepare Ascend NPU environment

Dependencies

Requirement Supported version Recommended version Note
Python >= 3.9 3.10 Required for vllm
CANN >= 8.0.RC2 8.0.RC3 Required for vllm-ascend and torch-npu
torch-npu >= 2.4.0 2.5.1rc1 Required for vllm-ascend
torch >= 2.4.0 2.5.1 Required for torch-npu and vllm required

Below is a quick note to install recommended version software:

Containerized installation

You can use the container image directly with one line command:

docker run \
    --name vllm-ascend-env \
    --device /dev/davinci1 \
    --device /dev/davinci_manager \
    --device /dev/devmm_svm \
    --device /dev/hisi_hdc \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
    -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
    -v /etc/ascend_install.info:/etc/ascend_install.info \
    -it quay.io/ascend/cann:8.0.rc3.beta1-910b-ubuntu22.04-py3.10 bash

You do not need to install torch and torch_npu manually, they will be automatically installed as vllm-ascend dependencies.

Manual installation

Or follow the instructions provided in the Ascend Installation Guide to set up the environment.