Most companies do not keep their knowledge in neat, ready-made datasets. It lives in contracts, invoices, product specifications, quarterly reports, and in that one spreadsheet the finance team has maintained for six years. When you decide to fine-tune a model on your own data, the pile of PDFs and spreadsheets is usually where you start — and it is also where most projects quietly go wrong. The model is rarely the problem. The preparation is.
Why raw files are not training data
A PDF is a layout format, not a data format. It stores instructions for drawing a page, and only incidentally the text itself. Copy text out of a two-column PDF and you will often get lines from both columns interleaved; extract a table and you may get every cell glued into one unbroken character soup. Spreadsheets look structured, but they hide their own traps: merged cells, formulas that no longer match the displayed values, mixed date formats, and workbooks where the same column means something different on tab one and tab four. Train directly on this material and your model inherits every one of those defects — then reproduces them confidently.
Extract first, clean second
Get everything into plain, structured intermediate files before you touch quality at all. The right extraction route depends entirely on the source:
- Digital PDFs: pull the text layer with a parsing library, then check reading order and table boundaries on a sample of pages before processing the rest of the batch.
- Scanned PDFs: run OCR first and expect to review low-confidence pages manually — recognition errors left in the output become training errors.
- Spreadsheets: export values rather than formulas, unmerge every merged cell, and treat each sheet as a separate table until proven otherwise.
- Whatever route you take, verify that nothing was dropped silently. Compare page, table, and row counts against the source files before moving on.
The cleaning checklist
Cleaning is where most of the real hours go, and it is repetitive by nature. Work through this list for every batch:
- Strip page furniture: headers, footers, page numbers, and the legal boilerplate repeated on every page.
- Normalise characters so curly quotes, multiple dash styles, and odd encodings each collapse to one canonical form.
- Rebuild structure: one row should mean one record, combined columns should be split apart, and every column needs a name a stranger could understand.
- Standardise dates, currencies, and numeric formats across the entire set.
- Deduplicate aggressively — repeated contract clauses and copy-pasted rows bias a model toward repetition.
- Mark missing values explicitly instead of leaving empty strings, so the model can learn what absence looks like.
- Remove or flag anything you would not want the model to reproduce: personal data, credentials, confidential terms.
From documents to training examples
A fine-tuning run never sees your documents — it sees examples. For instruction tuning, that usually means converting each passage into a prompt–response pair: a question a real user would ask, and an answer grounded strictly in that passage. Long documents should be split into chunks that each stand on their own without needing the surrounding pages. For spreadsheets, define the prediction target before formatting anything: are you classifying rows, generating summaries, or answering questions against the table? The answer changes the shape of every example you build.
Write a short labelling guideline before the first example, not after the hundredth. It does not need to be long — a single page defining the format, the tone, and how to handle uncertainty will keep example four hundred consistent with example four.
Quality control before you press train
Before committing weeks of compute, spend one day auditing. Sample twenty examples at random and read them against the source documents; you are checking extraction fidelity and label quality at the same time. Hold out a small validation set drawn from documents or rows the model will never see during training, and resist the temptation to peek. Confirm that the same kind of question always produces the same kind of answer, whoever labelled it. If the sample reveals systematic problems, fix the pipeline rather than the individual examples — patches at the source propagate everywhere.
None of this is glamorous, but preparation is the highest-leverage phase of a custom-model project. A modest model trained on five hundred clean, consistent examples will outperform a much larger one trained on fifty thousand rows of raw, inconsistent exports. The file formats were never the point. The point is giving the model something true, consistent, and well-shaped to learn from.
