[Usage]: 为什么CPU KV cache usage一直为0.0%

[WangJianQ-0118]

Your current environment

The output of `python collect_env.py`

How would you like to use vllm

I want to run inference of a [specific model](put link here). I don't know how to integrate it with vllm.

我用的是L40，发现GPU KV cache在使用中，但是CPU KV cache一直为0，想利用起这部分资源，请问大佬们该如何配置呀？

INFO 01-09 09:53:39 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 18.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 20.0%, CPU KV cache usage: 0.0%.
INFO: 10.38.115.100:51522 - "GET /metrics HTTP/1.1" 200 OK
INFO 01-09 09:53:44 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 18.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 20.0%, CPU KV cache usage: 0.0%.
INFO: 10.38.115.100:51522 - "GET /metrics HTTP/1.1" 200 OK
INFO 01-09 09:53:49 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 18.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 20.0%, CPU KV cache usage: 0.0%.
INFO: 10.38.115.100:51522 - "GET /metrics HTTP/1.1" 200 OK
INFO 01-09 09:53:54 metrics.py:351] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 18.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 20.0%, CPU KV cache usage: 0.0%.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.