Two terms come up almost every time someone gets serious about using AI at work: tokens and context windows. Both sound technical, but the ideas behind them are straightforward — and understanding them changes how you use AI tools more effectively.
What is a token?
AI language models don't read text the way you do — one word at a time. They break text into smaller pieces called tokens.
A token is roughly a word fragment. "Unbelievable" might be three tokens. "cat" is one. Spaces, punctuation, and common short words each count separately too. As a rough rule of thumb, 100 words of English is around 130–150 tokens — though the exact number varies by language, model, and the content itself.
Why does this matter? Because most AI services charge by the token, and because models have limits on how many tokens they can process at once.
What is a context window?
The context window is the total amount of text a model can hold in its attention at one time. Everything in the conversation — the system instructions, your messages, the model's previous replies, any documents you've pasted in — all of it counts against the context window.
Think of it like a desk. Everything you need to work on has to fit on the desk. If you pile on more than fits, things fall off the edge — in model terms, the oldest content gets dropped, and the model simply can't refer to it any more.
Why context window size matters
Context window size has grown dramatically over the past few years. Early production models had windows of 4,000 tokens — roughly a long document. Current leading models from Anthropic, Google, and OpenAI offer windows of 100,000 to 1 million tokens, enough to hold entire codebases or lengthy reports.
Larger windows matter for several real use cases:
Long document analysis. Pasting an entire contract, research paper, or transcript and asking questions about all of it.
Multi-turn conversations. Deep research sessions or extended coding help where you need the model to remember context from early in the conversation.
Code review. Passing a full codebase rather than individual files, so the model understands how the pieces interact.
Summarisation. Processing an entire book, set of earnings calls, or dataset in one pass.
The cost side
Because services charge per token, context window usage has a direct effect on cost. Longer conversations cost more. Pasting a large document costs more. And because input tokens (what you send) and output tokens (what the model generates) are usually priced differently — input is typically cheaper — the shape of your usage affects the bill.
For teams building with AI APIs, understanding token usage is essential for cost management. For individual users, it mostly means being thoughtful about what you include in a conversation and clearing context when starting a new, unrelated task.
Tokens in practice
A few practical points worth knowing:
- Non-English languages often use more tokens per word than English, because less common character sequences are broken into more pieces.
- Code tends to be token-efficient relative to its information density, because common syntax gets compact representations.
- Images are converted to token equivalents in multimodal models — a typical image can cost several hundred tokens.
- Structured data (JSON, CSV) can be verbose in token terms; consider summarising before passing large tables.
For a deeper look at how the underlying models work, see what is a large language model. For how multiple models can be compared and selected, see how AI model routing works.
On the askFinz platform, token usage and context handling are managed automatically across every model — you get the benefit of large context windows without having to track limits yourself. Get early access.