[QST] CUDA driver version and runtime version mis-match

[RuokaiYin]

What is your question?
Hello,

I have followed the Quick Start documentation and managed to build the project (using arch). When I went on to build for the test files, there was this CUDA mis-match error:
Error: Could not detect active GPU device ID [CUDA driver version is insufficient for CUDA runtime version]

I attached my CUDA versions below:

nvcc -V

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Mon_Apr__3_17:16:06_PDT_2023
Cuda compilation tools, release 12.1, V12.1.105
Build cuda_12.1.r12.1/compiler.32688072_0

nvidia-smi

Thu Sep  5 14:53:58 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.42.02              Driver Version: 555.42.02      CUDA Version: 12.5     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX A5000               On  |   00000000:17:00.0 Off |                  Off |
| 30%   23C    P8             18W /  230W |       2MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA RTX A5000               On  |   00000000:CA:00.0 Off |                  Off |
| 30%   23C    P8             16W /  230W |       2MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Is the mismatch referring to the CUDA Version 12.5 from nvidia-smi and the cuda_12.1.r12.1 from the nvcc -V? Does this mean that I need to get the CUDA with version of 12.5? I am very new to the CUDA coding, sorry in advance for the silly questions.. Thank you very much!

[thakkarV]

that's right, your CUDA toolkit version is too old for the driver you have installed on your system. If you upgrade CTK to 12.5 it should work fine

[RuokaiYin]

Thank you very much for the quick response! I will first try to update the CTK to 12.5.