Why “it looks good” is not a test
Every team that fine-tunes a model reaches the same moment: the outputs look better than they did last week, so someone suggests shipping. That instinct is understandable and risky. Fine-tuning changes the shape of a model's behaviour in ways that are invisible across ten hand-picked examples. The refusal that used to be polite becomes curt. The model that rarely invented facts on your test questions starts inventing them on questions phrased slightly differently. Evaluation exists to catch those shifts before your users do.
Build a golden set before you train
The most valuable artefact in a fine-tuning project is not the weights — it is a set of questions with known-good answers that you never train on. Assemble it from real usage wherever possible: support tickets, chat logs, the questions customers actually ask. A usable starting point is 80 to 150 examples covering the full range of what the model will face, with the hard cases deliberately over-represented.
- 20–30 straightforward cases the model must get right every time.
- 20–30 ambiguous cases where a reasonable answer needs hedging or a clarifying question.
- 10–15 cases the model should refuse — out of scope, unsafe, or confidential.
- 10 cases phrased in your users' own messy language, typos included.
- 5–10 cases with no good answer at all, to see how gracefully it fails.
Freeze the set and version it
Write your golden set down, commit it, and treat changes to it as a decision rather than housekeeping. If you edit questions after seeing model output, you have quietly moved the goalposts. When you must change the set, record the new version and re-report every model against both. Two numbers from the same set are comparable; two numbers from different sets are not.
Score behaviour, not vibes
Automated metrics catch regressions cheaply. Human review catches the rest. Use both, and keep the categories separate so you can tell what moved.
- Exact and fuzzy match on questions with a single defensible answer.
- Rubric scoring for open answers: accuracy, completeness, tone, formatting.
- Refusal rate on cases that should be declined — both missed refusals and over-refusals.
- Format compliance if your application parses the output into JSON, tables or fields.
- Blind human review of 30–50 answers per candidate, labelled without knowing which model produced them.
If you can pay for it, run a second model as a judge and measure agreement with your human reviewers. A judge that agrees with people 85% of the time is useful; one that agrees 55% of the time is noise with a confident tone.
Stress-test the edges
Fine-tuning narrows a model's world. Push on the boundaries deliberately: prompt injection attempts, questions in a second language, very long inputs, empty inputs, requests for things you explicitly do not support, and the same question asked five different ways. Record where behaviour becomes unpredictable, because those are exactly the places production traffic will find.
Always compare against the base model
A fine-tune that beats your previous fine-tune but loses to the untouched base model is not progress. Run the base model, your candidate, and the current production model through the identical golden set on the same day. Report the three scores side by side. This single practice prevents more bad releases than any other.
Measure the numbers the business already tracks
Accuracy is an internal metric. Escalation rate, resolution time, and the share of answers a human had to rewrite are the ones your stakeholders recognise. Map your evaluation scores to at least one of them, so the release conversation is about outcomes rather than percentages.
Put a gate in front of the release
Decide before you look at results what “good enough” means. A workable gate for a first release:
- No regression larger than 3 points on the golden set versus the current production model.
- Refusal behaviour at or better than the previous version in both directions.
- Human reviewers prefer the candidate on at least half of the blind comparisons.
- Latency and cost within the budget the product team agreed to.
If the candidate misses the gate, do not negotiate with the numbers. Go back to the data — the failure pattern usually points directly at what your training set was missing.
Make it a routine, not an event
The teams that ship confidently are the ones that run this loop every week, even when nothing has changed. Thirty minutes of automation plus a short review meeting turns evaluation from a launch-week panic into a background signal. That is the whole trick: you are not testing a model once, you are building the habit of knowing what it does.
