Inference after QLoRA fine-tuning

sade-adrien
Tue, 07 Nov 2023 17:05:55 +0000

Apparently, PEFT v.0.6.0 which was released a couple of days ago is now able to to merge LoRA weights with 8bits base models. Should look more into it to make sure it’s done right, but if we have faith it should be fine. Haven’t tried it yet tho