AutoModelForCausalLM error with accelerate and bitsandbytes

marcsun13
Mon, 15 Apr 2024 13:31:17 +0000

Hi @altrastorique, bnb quantization requires you to have cuda or cudnn to work. The error that you are facing is probably due to that.

Read full topic