https://disobey.dev/tags/llama.cpp/Recent content in Llama.cpp on disobey.devHugo -- 0.152.2en-usFri, 18 Sep 2026 00:00:00 +0000https://disobey.dev/posts/ternary-bonsai-27b-16gb-vram/Fri, 18 Sep 2026 00:00:00 +0000https://disobey.dev/posts/ternary-bonsai-27b-16gb-vram/<p>The internet is losing its mind over &ldquo;1-bit&rdquo; models again, so when I saw <a href="https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf?ref=disobey.dev">Ternary Bonsai 2 27B</a> sitting there as a 5.9GB gguf, the RTX A5000 in my laptop had to earn its keep again. Same 16GB card that <a href="https://disobey.dev/posts/gemma-4-26b-262k-context-16gb-vram/">carried the Gemma 4 experiment</a>, same question: how much of this thing can I actually use.</p> <p>Spoiler for the impatient: PQ2_0 packing, 128K context, ~30 tok/s on short prompts, 22 tok/s after a 65K-token prompt.</p>