Every time you send a message to an AI model, the model reads before it writes. Your prompt, the conversation so far, any documents you pasted, the standing instructions behind the scenes — all of it has to fit into one working space called the context window. It is the least glamorous specification in AI, and one of the most consequential.
Understanding the context window explains why an assistant sometimes forgets what you said ten minutes ago, why pasting a hundred-page PDF can fail or cost a surprising amount, and why two models with similar benchmark scores can behave very differently on real work.
What the context window actually is
A context window is the maximum amount of text a model can take into account in a single request, measured in tokens. Tokens are small fragments of text — roughly three quarters of an English word on average. A model with a 128,000-token window can consider about that many tokens of combined input and output. Everything the model knows about your situation comes from what sits inside the window at that moment. Outside it, the model has only its general training: nothing about your files, your earlier messages, or yesterday's conversation.
A useful mental model is a desk. The window is the surface of the desk, and the model can only work with the papers laid out on it. Anything in the filing cabinet across the room — your knowledge base, last week's chat log — does not exist until you place it on the desk.
Tokens, not words
Context is measured in tokens rather than words because models process sub-word pieces. In English, 1,000 tokens correspond to roughly 750 words. Tokenisation is not uniform: code, numbers and non-English languages often consume more tokens per idea, so a window that fits a twenty-page English report may hold far less of a Japanese contract or a dense spreadsheet. Treat the word equivalent of any quoted window as an approximation, not a promise.
What actually fits
The window is shared. Your prompt, retrieved documents, conversation history and the generated answer all draw from the same budget. Rough English equivalents for common sizes:
- An 8,000-token window holds about 6,000 words — a long essay or a short meeting transcript.
- A 32,000-token window holds about 24,000 words — a novella or a sizeable product manual.
- A 128,000-token window holds about 96,000 words — a typical novel or hundreds of pages of documentation.
- A 1,000,000-token window could take in an entire codebase or months of chat history — if you can afford to send it.
The generated reply also consumes the budget, so the practical space for your material is always a little smaller than the headline number.
Why bigger is not automatically better
Large windows solve real problems, but three trade-offs come with them.
- Cost and speed: most providers charge per token, so longer inputs are more expensive and slower to answer.
- Attention quality: models tend to recall the start and end of a long input better than the middle, so vital details buried on page forty can be underweighted.
- Signal-to-noise: a window stuffed with everything remotely relevant invites the model to pull in the wrong things, which hurts accuracy more than a smaller, curated set.
A long context window is a bigger workbench, not a warehouse. Memory that persists across sessions is a separate feature, built by the product around the model rather than by the window itself.
Using the window well
Getting good results has less to do with the size of the window and more to do with what you put in it.
- Retrieve, do not dump: use search or retrieval to place only the relevant sections of your documents into the prompt.
- Summarise and roll forward: replace old conversation turns with a short summary so recent work keeps the space it needs.
- Front-load or end-load what matters: instructions and key facts at the beginning or end of the prompt are typically followed more reliably than those buried in the middle.
- Watch usage: log token counts per request so a quietly growing prompt does not inflate cost and latency unnoticed.
How much window do you need?
Match the window to the workload. Everyday chat and drafting rarely need more than a small window. Question answering over long documents sits comfortably in the mid-range when retrieval is selective. Whole-codebase reasoning, long transcripts and multi-document analysis justify the largest windows. Paying for headroom you never use is a tax on every request, while buying too little shows up as truncation errors and forgotten instructions.
The context window is where the thinking happens. Teams that treat it as a scarce, shared resource — curating what goes in, summarising what stays, and measuring what it costs — get sharper answers and smaller bills than teams that simply chase the biggest number on the spec sheet.
