Fine-Tuning & Adaptation

What it is

Fine-tuning takes a pretrained LLM and continues training it on a smaller, task-specific dataset, updating the model's weights so its behavior shifts toward that task — distinct from prompting (which changes what you ask, not the model itself) and RAG (which changes what context the model sees, not its weights).

Key approaches

• Full fine-tuning: updating all of the model's parameters — most flexible, but requires substantial compute/memory and risks catastrophic forgetting (the model losing general capabilities while specializing).
• Parameter-efficient fine-tuning (PEFT): methods like LoRA (Low-Rank Adaptation) freeze the original weights and train a much smaller set of additional parameters — dramatically cheaper and faster than full fine-tuning, with results competitive for many tasks.
• Instruction tuning: fine-tuning on examples of (instruction, ideal response) pairs specifically to make a base model better at following instructions in a helpful, consistent format.
• RLHF / preference tuning: further aligning a model's outputs to human preferences by training on comparisons between candidate responses, rather than single "correct" examples — this is what shapes tone, helpfulness, and refusal behavior beyond raw instruction-following.

When to fine-tune vs. prompt or RAG

• Fine-tuning is the right call when the need is a behavior or format change that's hard to reliably get through prompting alone (e.g. consistently outputting a specific structured format, or adopting a specific tone at scale) — not for injecting knowledge, which RAG handles far more cheaply and updatably.
• A common mistake is reaching for fine-tuning to add domain knowledge — RAG is almost always cheaper and easier to keep current for that use case, since fine-tuning bakes knowledge into weights that go stale the moment the underlying facts change.
• Fine-tuning requires a real training dataset (typically at least hundreds to thousands of quality examples) — for tasks where that data doesn't exist yet, a well-crafted prompt (or few-shot examples in-context) is usually the faster first thing to try before investing in fine-tuning.