https://disobey.dev/tags/self-hosted/Recent content in Self-Hosted on disobey.devHugo -- 0.152.2en-usWed, 03 Jun 2026 00:00:00 +0000https://disobey.dev/posts/gemma-4-26b-262k-context-16gb-vram/Wed, 03 Jun 2026 00:00:00 +0000https://disobey.dev/posts/gemma-4-26b-262k-context-16gb-vram/<p>I&rsquo;ve been running local LLMs for a while now and the eternal struggle is always the same: you want more context, more model, more speed — and you have none of the VRAM to support any of it. So when Gemma 4 dropped with 262K context window I obviously had to try fitting the whole thing on my RTX A5000. 16GB. Turns out you can. And its actually usable.</p> <p>~658 tok/s prompt eval. ~35 tok/s decode. Full 262K context window. f16 KV cache, no compression tricks.</p>