[Official FAQ] Share your experience or follow-up questions below.

Local models can be less consistent than large hosted ones at following formats, but a few techniques help a lot.

• Use constrained or structured output: Ollama accepts a JSON schema through its format option, llama.cpp supports grammars and JSON-schema constraints, and vLLM offers structured (guided) decoding. These restrict which tokens can be generated, which is far more reliable than prompting alone. Exact parameter names depend on your version, so check the docs.
• Write a clear prompt: describe each field, include one complete example, and say "output only JSON".
• Lower the temperature to reduce random deviations.
• Validate in code: use Pydantic or a JSON Schema validator, and on failure retry once with the error message. Never assume the output is valid.

About tool / function calling

• Not every model supports it. Pick a model whose card says it does, and make sure your runtime applies the correct chat template.
• Keep tool descriptions short and precise, and avoid offering too many tools at once, especially to small models.
• For tools with side effects (deleting files, sending messages, running commands), enforce permissions in your own code and consider human confirmation.

Small models struggle with deeply nested output, so break complex tasks into smaller steps.

What format problems have you hit? Share a prompt and sample output and we can suggest fixes.