[Official FAQ] Corrections and follow-up questions welcome.

Public scores help, but prompt formats and domains differ. Build a small private set (tens to low hundreds of items) from real tasks.

Method:

• Freeze templates, temperature, and decoding settings when comparing.
• Score correctness, refusals, formatting, latency, and peak memory separately.
• Optional harnesses/scripts can automate scoring; LLM-as-judge adds its own bias.

Changing quant or chat template can swing scores—control variables. Share anonymized rubric structures if useful.