[Daily Picks]

r/LocalLLaMA is circulating a freshly merged llama.cpp change (PR #31356): Gemma 4’s small E2B/E4B models were still running full compute on layers that only reuse earlier KV and never write their own cache. The fix prunes unused prompt rows before those KV-shared layers (unified-KV path) and also simplifies the PLE projection.

On an M4 Pro (Metal) with Google’s QAT Q4_0 builds, reported prompt-processing jumps are huge—E2B pp512 roughly 1.6k → 4.6k tok/s, E4B about 0.8k → 1.3k tok/s—while token generation stays basically flat. Similar wins show up on Snapdragon HTP. The 26B dense path doesn’t benefit the same way because every layer feeds KV. Outputs were checked identical across text/image/audio paths in the PR notes.

Practical takeaway: if you use Gemma 4 E2B/E4B for long-context RAG or agent prefill, update llama.cpp (or whatever ships a recent build) and re-bench pp before you blame the model. Split-KV currently skips the prune, so unified KV matters for the speedup.

Are you on E2B/E4B locally—what pp numbers do you see after updating?

Source: https://www.reddit.com/r/LocalLLaMA/comments/1xn6mm0/gemma4_graph_optimizations_massive_gains_on_pp/