Context window

A context window is the amount of information an AI model can work with in one request. The task, instructions, chat history, files, tool information, and answer all need room in that working space.

A bigger window can help when the job needs a lot of material, but fitting a file doesn’t mean the model will find every important fact in it. For client work, I’d check whether it can find the current rule, approved price, and client’s limits when the request runs, rather than whether it accepts the whole archive.

What uses the window?

Almost everything the model receives and produces uses part of the window. Google’s token guide describes the context window as the combined input and output limit. Its usage records count the prompt, system instructions, conversation history, cached material, tool use, and generated answer. Images, audio, and video also use tokens after processing.

Think of the window as the packet for one job. It may contain standing instructions, the client’s brief, three approved examples, an email thread, notes returned by a search tool, and earlier turns in the chat. The answer needs room in that same packet.

That’s why a chat can reach its limit before the conversation you see looks very long, the app may also be sending instructions, files, and tool definitions. When the packet is full, the app has to reject new material, shorten or summarize the history, or remove older parts, depending on how the product works.

A token is a small unit the model processes, not the same as a word. You don’t need to count them by hand, just remember that each extra policy, transcript, and old draft uses some of the room.

Why can a fact fit and still be missed?

A fact can fit inside the window and still be hard for the model to recover. That is the key limit of capacity alone.

In 2024, Nelson Liu and co-authors published Lost in the Middle in the Transactions of the Association for Computational Linguistics, testing models on questions using several documents and on tasks that required finding facts. Performance was often highest when the relevant fact appeared near the beginning or end, and fell when the same fact moved into the middle.

Google Research reported a similar U-shaped bias in 2024. The researchers found that models paid more attention to the beginning and end of long input even when those parts were not the relevant ones.

Imagine a proposal thread that starts with the approved price, then collects six drafts, two meeting transcripts, copied research, and a late scope change. The correct price still fits, but the model could use a number from draft four because several plausible prices are now mixed together.

A reviewer might see a confident total and assume the model had all the information it needed. It may have had that information and still used the wrong part, which is why you need to test the answer, not just the upload.

Is a bigger window better?

A bigger window helps when the job needs more source material, such as comparing a long contract, tracing project decisions, or reviewing many call transcripts without splitting them first. Google’s long-context guide says many Gemini models accept 1 million or more tokens, with 1 million roughly enough for 50,000 lines of code, eight average English novels, or more than 200 average podcast transcripts.

Those are size comparisons, not promises that the model will use everything equally well.

More room also brings three costs.

First, the firm pays to process more input. Prices vary by model and change over time, but sending the same 300-page manual on every request costs more than sending the three pages that govern the task.

Second, longer input takes longer to start answering. Google’s guide says longer queries generally increase time to the first token.

Third, the model may become less reliable at finding facts when the request asks for more at once. Google says a model may reach about 99% on a single fact-retrieval query while performance varies widely when several facts must be found in the same context. Its advice is to omit tokens that are not needed.

I’d choose enough room for the current brief, approved rules, and relevant examples, then fetch other material when the job needs it. The client archive can stay in storage, you don’t have to put it all in every request.

Context window, context, and memory

These terms name different parts of the system.

Context is the information available for the current request, and the context window is the room available for that information and the answer. Memory is information saved outside the request and brought back later.

A client preference stored in a CRM becomes context when the application retrieves it and places it in the current request. Once it enters, it uses the context window like every other instruction or document.

That’s why AI can use a fact in one conversation and miss it in another. The information may still be in storage without being fetched, it may have been dropped from the old thread, or it may still be present but buried, and each of those needs a different fix.

Context engineering manages the selection. It decides what belongs in the packet, what gets fetched later, and what is removed when it becomes stale. The context window only tells you how much room exists.

How should you manage the window in client work?

Keep a clean working packet for each repeated job. Four habits cover most situations.

Start a fresh thread when the job changes. A proposal, delivery review, and renewal decision should not share one endless conversation. Old turns create plausible material that belongs to another decision.

Put current rules before reference material. Give the model the approved pricing rule, scope boundary, and output format before transcripts and examples. Put the request after the source material. Google’s guide recommends placing the query at the end of a long prompt.

Replace stale material. Do not append a corrected policy below the old one and expect the model to choose. Remove the old version. The same rule applies to superseded briefs and rejected drafts.

Test fact recovery. Before a workflow touches client work, ask questions whose answers sit at the beginning, middle, and end of the packet. A model accepting a file proves only that the file fit. The real test is whether it recovers the facts the work depends on.

That’s why documenting the process first matters. If nobody has written down which rules and records govern the job, the workflow has a selection problem before it reaches the model’s size limit.

Where does this sit on the Delivery Model Ladder?

A context window matters at every stage, but who prepares the information changes. At Stage 1, Enhanced, on the Delivery Model Ladder, a person assembles the packet and asks the model to handle more of the task, so a larger window can help them without changing the delivery process itself.

Stage 2 starts when an AI workflow selects the client record, current policy, and relevant examples on its own. Context stops being whatever someone happened to paste. It becomes part of process design.

That handoff is part of the Production Gap. A demo works because its creator knows which five documents to upload. Production works only when the system can choose the right five on the hundredth run, after two policies change and another quarter of client email arrives.

A larger window doesn’t settle which source to trust, supply a missing rule, or prove the answer is correct. Give the model the current information for the job and test whether it can find and use it.

Quick answers

Does AI remember everything in a long chat? No. Earlier messages use the same finite window as the current request. The product may drop or summarize older material, and information that remains can still be harder to recover in the middle.

What happens when the context window is full? The application must refuse more input, remove older material, shorten it, or start a new request.

Does a larger context window make AI more accurate? It gives the model access to more information. Accuracy still depends on whether the right information was supplied, whether stale material conflicts with it, and whether the model can recover the facts the task needs.

How much context should I provide? Give it the current instructions and facts, with enough examples to show the standard you’re asking for. Leave the rest of the archive in storage until the job needs something from it.