Versioning a model is not like versioning code
Code diffs are small and readable; a model artefact is a binary blob whose behaviour depends on three things at once: the base weights, the data it saw, and the configuration used during training. Changing any one of them produces a model that behaves differently, even when the file size looks identical. That is why "v2" tells you almost nothing. A useful version answers a harder question: if this model misbehaves at 3 a.m., can someone reproduce exactly how it was made and swap it for the last known-good build without improvising?
What a version must capture
Treat every trained model as a bundle, not a file. A complete version captures four layers together:
- The artefact. The final weights or the adapter (for LoRA-style training), plus a checksum so you can verify what actually shipped.
- The data. A pointer to the exact dataset snapshot — not "the October folder", but an immutable reference with its own hash.
- The recipe. Training configuration: hyperparameters, framework versions, seed, and the training code commit.
- The evidence. Evaluation results that justified promoting this build — accuracy, latency, and the failure cases you already know about.
If any layer is missing, your "rollback" later becomes archaeology: people dig through chat threads trying to remember which dataset produced the model that worked.
Naming that survives production
Keep it boring and ordered. A scheme like assistant-2026.10.3 (project, year, month, build) sorts correctly in every file browser and immediately answers "which is newer?" Add a short tag for the reason: -hotfix, -data-refresh, -eval-passed. Reserve special meaning for exactly two labels: production — what users are hitting right now — and stable — the last build you would happily fall back to. Everything else is just a candidate. The moment you let "latest" mean "probably fine", you have lost the plot.
Where versions live
You do not need elaborate tooling on day one. A model registry can start as three things: object storage holding the artefacts, a small metadata table (a spreadsheet is genuinely acceptable at small scale) mapping version to data snapshot, config, checksum, and eval scores, and one directory on the serving host that always reflects what is deployed. What matters is the discipline, not the brand of registry: one place that answers "what is in production, and what came before it?" Dedicated registries earn their keep when you have multiple models, frequent releases, or several people promoting builds — migrate then, not before.
Rollback: the plan you make before the incident
Rollback fails in practice for unglamorous reasons: the old artefact was deleted to save storage, the previous serving config was never recorded, or nobody knows how to point traffic back. Design the rollback path as deliberately as the rollout:
- Keep at least the last two production artefacts immediately loadable; archive the rest, never delete blindly.
- Pin serving-side dependencies. A rollback that needs a library upgrade first is not a rollback.
- Make the switch a configuration change, not a rebuild — an environment variable or a registry pointer, applied in minutes.
- Write the runbook while calm: exact commands, who approves, and how to verify users are seeing the old model again.
Make rollback boring with a staged rollout
The cheapest rollback is the one that catches problems early. Ship new versions behind a small traffic slice first — five percent for a day, watched with the same evaluations you ran offline plus a live canary metric (deflection rate, escalation rate, whatever matters in your domain). Compare against the incumbent model, not against hopes. If the canary degrades, flipping the flag back costs nothing. If it holds, ramp gradually. This turns rollback from an emergency procedure into a routine toggle — which is precisely the point.
Versioning is unglamorous work, and that is its value. Teams that treat each model as a fully documented, instantly swappable release unit turn bad model days into ten-minute inconveniences instead of multi-day outages. Build the bundle, tag the build, and keep the exit open.
