Fine Tuning LLama 3.2 1B Quantized Memory Requirements

John6666
Mon, 16 Jun 2025 14:52:43 +0000

As BenjaminB also mentioned, it seems unusual that this model size and training parameters require this amount of RAM.
Therefore, there may be issues with the library or model, and the cause may lie in parts other than the visible parameters.

However, if this were a common issue, there would likely be many reports of it. It may be specific to certain conditions, such as a multi-GPU environment, a specific version of the library, or specific hardware.

GitHub

Fine tuning GPU memory requirements · ggml-org llama.cpp · Discussion #2904

This question is more focused on on full fine tune memory requirements rather than low memory / efficient inference, but I'm hoping it'll be relevant / helpful to community members here especially ...