LLM Fine-Tuning Interview Questions

LLM fine-tuning interview questions#

Fine-tuning questions are common in AI engineer and ML engineer interviews. Interviewers look for practical judgement — knowing when not to fine-tune is as important as knowing how.

1. What is fine-tuning an LLM?#

Continuing to train a pre-trained model on a smaller, task-specific dataset so its behaviour changes — format, style, or a specialised task — while keeping its general language ability.

2. When would you choose fine-tuning over prompting or RAG?#

Prompting first, always. Choose RAG when the model needs private or changing knowledge. Choose fine-tuning for consistent style or strict formats, narrow repeated tasks, shrinking long prompts, or getting a smaller model to match a bigger one on one task.

3. Why is fine-tuning a bad way to add knowledge?#

Facts learned through fine-tuning are hard to update, hard to verify and can increase confident hallucinations. RAG keeps knowledge in documents that can be updated and cited.

4. Explain LoRA.#

LoRA freezes the pre-trained weights and learns a low-rank update for selected layers: two small matrices A (r × k) and B (d × r) whose product is added to the frozen weight. With a small rank, it trains well under 1% of the parameters, needs far less memory, and produces small swappable adapters.

5. What is QLoRA?#

LoRA applied on top of a base model quantized to 4-bit NF4, with double quantization and paged optimizers. It cuts memory enough to fine-tune multi-billion-parameter models on a single GPU, with quality close to standard LoRA.

6. What do r and lora_alpha control?#

r is the rank — the adapter's capacity. lora_alpha scales the adapter's contribution (by α / r). A common start is r = 16, alpha = 32.

7. Which layers do you apply LoRA to?#

Usually the attention projections (query, key, value, output) and often the MLP projections. Targeting more layers tends to improve quality for a small memory cost.

8. What learning rate would you use for LoRA?#

Higher than full fine-tuning — around 1e-4 to 2e-4 is common, with warm-up and cosine decay, typically for 1–3 epochs.

9. How much data do you need?#

It depends on the goal: a few hundred clean examples can teach a format; specialised tasks often need thousands. Quality and consistency matter more than quantity.

10. What is a chat template and why does it matter?#

Each instruction-tuned model formats conversations with its own special tokens. Training data must use that exact format (apply_chat_template in Transformers); a mismatch between training and inference is a frequent cause of poor results.

11. What is catastrophic forgetting and how do you limit it?#

The model loses general abilities while learning the new task. Limit it with parameter-efficient methods like LoRA, fewer epochs, lower learning rates, mixing in some general data, and checking a regression test set.

12. How do you detect overfitting during fine-tuning?#

Validation loss rises while training loss keeps falling. Respond with early stopping or an earlier checkpoint, fewer epochs, more or more diverse data, or higher dropout.

13. How do you evaluate a fine-tuned model?#

On a held-out test set, compared with a baseline (base model + best prompt). Use task metrics (accuracy, F1, valid-JSON rate), LLM-as-judge with a rubric for open-ended answers, and a regression set for general ability and safety.

14. Should you merge a LoRA adapter?#

Merge for simple deployment and zero extra latency. Keep adapters separate when you want to serve several tasks from one base model or swap adapters quickly. Merge into the full-precision base, not a 4-bit one.

15. What are SFT and DPO?#

SFT (supervised fine-tuning) trains on prompt → ideal response pairs. DPO (Direct Preference Optimization) trains on prompt → preferred vs rejected responses to align the model with human preferences, without training a separate reward model.

16. Your fine-tuned model performs worse than the base model. What do you check?#

Data quality and consistency, chat template mismatch, system prompt mismatch between training and inference, too high a learning rate or too many epochs, and whether the evaluation set really matches the task.

Practice tip#

Fine-tune a small open model on 300–500 examples of a narrow task, compare it with the base model on a held-out test set, and write up the results — including what didn't work. That project story answers most fine-tuning interview questions.