Very slow to load deep seekv3 int4 model and device_map="auto" "sequential" bug

[wenhuach21]

System Info

transforms 4.47.0

Who can help?

No response

Reproduction

please refer the code in model card https://huggingface.co/OPEA/DeepSeek-V3-int4-sym-gptq-inc

Expected behavior

1 Loading is very slow. Loading the model (https://huggingface.co/OPEA/DeepSeek-V3-int4-sym-gptq-inc) on a DGX system with 2TB of memory and 7x80GB A100 GPUs is very slow, taking 30 minutes to 1 hour.

2 device_map bug. Additionally, on a 7x80GB A100 GPUs, using device_map='auto' results in an OOM error, while switching to device_map='sequential' still causes an OOM error on card 0, even with max_memory configured.