[Official FAQ] Share your experience or follow-up questions below.

LoRA is a parameter-efficient fine-tuning method. Instead of changing all the weights, you train small added adapter matrices, which needs far less memory and time. QLoRA goes further by loading the base model in 4-bit, making fine-tuning of small and mid-size models possible on consumer GPUs.

Rough requirements (approximate)

• A 7B–8B model with QLoRA typically needs on the order of 10–20 GB of VRAM, depending on sequence length and settings. Larger models need more.
• Data: a few hundred to a few thousand high-quality examples can already change behavior. Quality beats quantity.
• Common tools: Unsloth, LLaMA-Factory, Axolotl, and Hugging Face PEFT with TRL.

Typical workflow: prepare instruction or chat-formatted data, pick an Instruct base model, train the adapter, evaluate on held-out examples, merge or load the adapter, then export (for example to GGUF) for local serving.

Fine-tuning or RAG?

• Need up-to-date or frequently changing knowledge: use RAG. You update documents instead of retraining.
• Need a consistent style, format, tone or task behavior: fine-tuning fits well.
• Fine-tuning is not a reliable way to inject lots of facts. The model may misremember them. Many projects combine both.

Cautions: do not train on sensitive data you would not want the model to reproduce, and always test on data it has not seen, since overfitting is common.

What would you like to fine-tune for? Describe your data and goal.