llama.cpp 0.9.0 Upgrade Guide: What Changed and How to Re-Benchmark Safely
Why this release matters
llama.cpp sits underneath a large share of the local LLM ecosystem. Ollama, LM Studio and many desktop apps either embed it or track it closely, so a tagged release is worth a careful look even if you never build it yourself. Version 0.9.0, tagged on November 4, 2026, bundles several changes that are relevant to people who run models on their own hardware. This guide summarizes the headline items from the release notes and, more importantly, explains how to upgrade without losing track of whether things actually got better.
Headline changes in plain language
FP8 E4M3 quantization. The release adds support for the OCP FP8 E4M3 format. FP8 is an 8-bit floating-point format that some newer GPUs handle natively. For users it means another option alongside the familiar integer GGUF quants. Whether it beats a Q8_0 or a K-quant on your card is an empirical question, so test rather than assume.
Multi-sequence KV rows. A new attention path is designed to make the unified KV cache friendlier when several sequences run at once. If you serve multiple users or parallel agent calls through llama-server, this is the item most likely to change your experience under load.
Batched tensor copy APIs. These are plumbing changes, but plumbing is often where throughput comes from. Expect the benefit to show up indirectly, through other components that adopt them.
Gemma 4 prefill work. Prompt processing for the small Gemma 4 E2B and E4B models continues to receive graph optimizations. If you use those models for long prompts or retrieval-heavy workloads, prefill speed is the number to watch.
New architectures and audio. The release lists support for several additional model architectures as well as text-to-speech and transcription components. Audio users may be able to drop a separate runtime and keep everything in one stack.
ggml update. The underlying ggml library moved to a new version with backend work across CUDA, Metal, Vulkan and SYCL, plus early multi-node tensor-parallel work.
A safe upgrade routine
- Record a baseline first. Before upgrading, run llama-bench or your usual script on the two or three models you actually use. Save prompt-processing and token-generation numbers along with the exact build hash, GPU driver version and command-line flags.
- Build into a separate directory. Keep the old binary around. If something regresses, you can switch back in seconds.
- Re-run the same benchmark. Use identical models, context sizes, batch settings and thread counts. Change one variable at a time.
- Check quality, not just speed. Run a small fixed set of prompts—a coding task, a summary, a tool-calling example—and compare outputs. Performance gains are worthless if a sampling or template change breaks your workflow.
- Test concurrency if you serve. For llama-server users, simulate the number of parallel slots you normally see. The multi-sequence changes are only meaningful under that kind of load.
Who should upgrade now
Upgrade promptly if you run Gemma 4 small models, serve several clients in parallel, or want to try FP8 on supported hardware. If you rely on a wrapper like Ollama, wait for that project to pick up the new version, since mixing versions by hand can cause confusing behavior. If your current build is stable and none of the headline items apply to you, there is no harm in waiting a week for early bug reports.
Common pitfalls
- Old benchmark numbers from forums were measured on different builds; do not compare them directly with yours.
- New flags sometimes change defaults. Read the help output after building.
- GPU driver updates done at the same time make it impossible to tell which change helped.
Bottom line
Version 0.9.0 is a meaningful release for local inference, especially for parallel serving and small Gemma models. Treat it like any infrastructure upgrade: baseline, isolate, measure, and keep a rollback path.
Join the discussion in our forum: read and reply to the matching forum thread.
