About the Author
A six-stage progression series showing how token usage evolves across a conversation — from fresh start through sliding window truncation.
Every conversation with a large language model fills a fixed container called the context window. Understanding how it fills — and what happens when it overflows — changes how you work with AI.
New conversation begins with only system instructions loaded. Maximum space available for interaction.
User sends first message. System processes input and generates a response, starting conversation history.
Multiple exchanges build conversation history. Each turn adds both user messages and AI responses.
User uploads a large document or code file. The input section expands significantly, consuming available space rapidly.
Context window is nearly full. The system must become selective about what information to retain going forward.
Context limit reached. Older conversation history is automatically removed to make room — the AI may lose context from early in the conversation.
Understanding the context window changes how you structure conversations, upload files, and design AI workflows.
System instructions and key reference documents occupy the first and most stable part of the window. Load the most important context early — not mid-conversation when space is already consumed.
A 2,000-line codebase or a long PDF can consume 60–80% of available context in a single turn. If you need to discuss multiple documents, start a fresh conversation for each to preserve working space.
The model won't announce when it begins truncating history. If responses start losing coherence in a long conversation, truncation may have already occurred. Starting fresh is often more productive than continuing.
When building agentic workflows, token budget management is an operational discipline. Agents that carry too much history per turn will hit limits sooner. Design prompts and memory structures to be compact without losing reasoning quality.
One practical response to context limits is the reference document approach — a single, dense markdown file loaded at the start of every session that gives the model the context it needs without requiring the conversation to rebuild it each time.
Modern context windows feel large but fill quickly in real work. A system prompt, an ecosystem document, a codebase, and a long conversation can exhaust 200K tokens in a single session. Plan for limits, not against them.