Ask ten teams what a custom AI model costs and you will get ten different numbers, because they are quoting different projects. A narrow fine-tune and a from-scratch pretraining run differ by orders of magnitude. The useful question is not "what does it cost?" but "where does the money go, and how do I control it?"
The four cost buckets
- Data preparation — cleaning, structuring and formatting your examples.
- Compute — the GPU hours spent training.
- Evaluation — the human and automated work that decides whether the model is good enough.
- Running costs — serving, monitoring and updating the model after launch.
Most newcomers over-weight compute and under-weight the other three, which is exactly where projects overrun.
Data preparation is usually the biggest line
If your knowledge lives in documents, spreadsheets and tickets, converting it into clean instruction/response pairs is skilled, patient work. This is the stage that decides quality, and it is billed in people, not GPUs. Scope it honestly and it will dwarf your training bill.
Compute is more affordable than it looks
Parameter-efficient methods such as LoRA mean you rarely need a large cluster. You adapt a strong open base on a modest GPU, often rented by the hour, and you can stop when the loss curve flattens. On-demand capacity that spins up for the run and shuts down afterwards turns a capital expense into a small operating one.
Evaluation is where trust is bought
A loss number is not a quality guarantee. Budget for a fixed evaluation set of real questions, scored before and after training, plus spot checks for regressions. Cheap evaluation is the most expensive decision in the project, because it is how bad models reach customers.
Running costs continue after launch
Serving has an ongoing cost: hosting, autoscaling, monitoring and periodic refreshes. A well-quantised model can run on modest hardware, and a smaller sibling can serve lighter traffic locally. Plan for this line from day one rather than treating launch as the finish.
How to keep the bill sane
- Start narrow: one task, one audience, one measurable success criterion.
- Fine-tune instead of pretraining unless you have a specific reason not to.
- Clean the data before you spend a single GPU hour.
- Reuse a strong open base rather than training behaviour from nothing.
- Ship retrieval first where it fits, and fine-tune only what remains.
The takeaway
The honest answer is that cost tracks scope. A focused fine-tune on prepared data is a modest, bounded project; an open-ended attempt to build a general model is not. Decide the scope, prepare the data, and the budget becomes predictable rather than open-ended.
