Two approaches dominate when teams build a domain-specific assistant: fine-tuning and retrieval-augmented generation (RAG). They solve different problems, and choosing the wrong one wastes weeks. The short version: fine-tuning teaches behaviour, retrieval supplies facts.
What each approach actually does
- Fine-tuning updates the model's weights on your examples. The result is a model that chats, formats, and reasons in your style — permanently.
- RAG leaves the model untouched and hands it relevant passages from your documents at answer time. The result is a model that can cite your latest information without retraining.
When fine-tuning wins
- You need a consistent tone, persona, or output format the base model will not hold reliably.
- Your task has a strict structure — a fixed JSON shape, a house writing style, a specific classification scheme.
- You want to shrink prompts and cost by baking instructions into the model rather than repeating them every call.
- Your knowledge is stable and you want it always present, not fetched.
When retrieval wins
- Your facts change often — prices, policies, inventory, regulations.
- You need citations and an audit trail back to a source document.
- You have a large knowledge base and cannot retrain every time a document changes.
- You want to ship in days, not weeks.
Why it is usually not either/or
The strongest production assistants combine both: a fine-tuned model that speaks and formats correctly, grounded by retrieval for the current facts. The fine-tune handles behaviour; retrieval handles truth that moves.
A quick decision checklist
- Does the problem change how the model behaves? → fine-tune.
- Does it require current or citeable information? → retrieval.
- Both? → fine-tune for style, retrieve for facts.
- Unsure, and you need to ship this month? → start with retrieval, measure, then fine-tune what remains.
Cost and time
Retrieval is usually faster to stand up and cheaper to maintain for changing knowledge. Fine-tuning carries a one-off data and training cost but can reduce per-call cost and improve reliability on structured tasks. Treat the choice as an engineering decision driven by your data and your tolerance for staleness.
