LLM Fine-Tuning Introduction
What fine-tuning an LLM means, full fine-tuning vs parameter-efficient methods, and how to decide whether you need it at all.
What is LLM fine-tuning?
Fine-tuning means continuing to train a pre-trained large language model on your own, smaller dataset so it behaves the way you need — a specific tone, a strict output format, a specialised task or domain vocabulary.
A pre-trained model already knows language. Fine-tuning doesn't teach it English from scratch; it adjusts its behaviour, the way an experienced developer adapts to a new team's coding style.
Fine-tuning vs prompting vs RAG
Before fine-tuning, always ask whether a cheaper method already solves the problem.
| Approach | What it changes | Best for | Cost & effort |
|---|---|---|---|
| Prompt engineering | The instructions you send | Most tasks — try this first | Lowest |
| RAG | The knowledge in the context | Private, large or frequently changing information | Medium |
| Fine-tuning | The model's weights | Consistent style/format, specialised skills, shorter prompts at scale | Highest |
A common misunderstanding: fine-tuning is a poor way to add facts. If your goal is "answer questions about our 2,000 policy documents", use RAG — documents change, and fine-tuned facts are hard to update and easy to hallucinate. Fine-tune for how the model answers, not what it knows.
When fine-tuning is a good idea
- You need a very consistent output format (e.g. always valid JSON for a specific schema, or medical-note structure).
- You want a specific voice or style that long prompts can't reliably produce.
- You have a narrow, repeated task (classification, extraction, routing) and want a smaller, cheaper, faster model to do it well.
- Your prompts have become very long with examples, and you pay for those tokens on every call.
- You must run on-premise or on-device with an open-weight model.
When it's not
- You haven't tried strong prompting with examples yet.
- You don't have at least a few hundred high-quality examples (often 500–5,000+ for good results).
- The knowledge changes weekly.
- You can't evaluate whether the result is actually better.
Types of fine-tuning
| Type | What's trained | Hardware | Notes |
|---|---|---|---|
| Full fine-tuning | Every weight | Many large GPUs for big models | Most flexible, most expensive, highest risk of forgetting |
| LoRA | Small low-rank adapter matrices | One GPU for many models | The default choice today |
| QLoRA | LoRA on a 4-bit quantized base model | A single consumer GPU, even for multi-billion-parameter models | Slightly slower training, huge memory savings |
| Hosted fine-tuning | Done by an AI provider's API | None on your side | Upload data, get a custom model endpoint |
LoRA and QLoRA — parameter-efficient fine-tuning (PEFT) — have made fine-tuning practical for students and small teams. You'll learn them in detail in this tutorial.
Kinds of training data
- Supervised fine-tuning (SFT) / instruction tuning — pairs of prompt → ideal response. This is what most projects need.
- Preference data — for one prompt, a preferred and a rejected answer, used by methods like DPO to align the model with human preferences.
- Continued pre-training — large amounts of raw domain text (legal, medical) to deepen domain language. Expensive; rarely the first step.
The fine-tuning workflow
- Define the goal and the metric — what does "better" mean, and how will you measure it?
- Build a baseline — measure the base model with your best prompt.
- Prepare data — collect, clean, format and split it.
- Train — usually LoRA or QLoRA on an open-weight model.
- Evaluate — compare against the baseline on a held-out test set.
- Iterate — most improvements come from better data, not bigger models.
- Deploy — serve the adapter or a merged model, and keep monitoring.
What you need to follow along
- Python and comfort with the command line.
- PyTorch basics (see the PyTorch tutorial).
- A GPU: a free cloud notebook GPU is enough for small models (0.5B–3B parameters) with LoRA or QLoRA.
- A Hugging Face account to download open-weight models.
Common mistakes
- Fine-tuning to inject knowledge. Use RAG for facts.
- Skipping the baseline. Without it you can't prove fine-tuning helped.
- Small, messy datasets. 200 inconsistent examples teach inconsistency.
- Testing on training examples. Always keep a separate test set the model never saw.
Key takeaways
- Fine-tuning adjusts a pre-trained model's behaviour using your examples.
- Try prompting and RAG first; fine-tune for style, format and narrow tasks.
- LoRA and QLoRA make fine-tuning affordable on a single GPU.
- Data quality and evaluation decide whether fine-tuning succeeds.