Squeezing Qwen 3.8 27B into a Single 16 GB GPU β€” Almost 42 tok/s, 64K Context

#5
by Ataa - opened

42 tokens/sec. 64K context. A 27B model. On a single 16 GB GPU.
I wanted to see how far I could push Qwen 3.8 27B on an AMD RX 9070 XT using llama.cpp.
So I tested:

β†’ 2 different quantizations
β†’ MTP2 vs. no MTP
β†’ 32K vs. 64K KV cache
β†’ The same real coding task across all configurations

The result surprised me:
πŸ”₯ 42 tok/s with MTP2
🧠 64K context actually fitting
πŸ’Ύ ~1 GB working RAM during evaluation

And MTP2 wasn't just a tiny optimization β€” it delivered +41% to +53% throughput depending on the quantization.
But there's a catch: the fastest configuration wasn't necessarily the best for reasoning quality.
I broke down all four configurations, commands, memory usage, CPU load, and the quality differences in my new benchmark.

πŸ‘‰ Full benchmark: ataadevs.net/blog/qwen-3-8-27b-llamacpp-benchmarks

Sign up or log in to comment