Fine Tuning LLama 3.2 1B Quantized Memory Requirements

rkapuaala
Mon, 16 Jun 2025 07:54:14 +0000

Has anyone fixed this yet? I’m using a AMD Ryzen 5 4600G with Radeon Graphics on my ubuntu server.
I’ve got 64GB of ram, 2 asus Geforce rtx ti 16 GB 4060 gpus and 4 1T nvme ssds. I still keep getting those errors. I’ve tried configuring accelerate at the begining of the script, setting every environment variable I can find, and the CUCA still demands 32 GB.