# What Is a Context Window in AI? Limits and Costs

URL: https://3ammarketer.com/blog/what-is-a-context-window-in-ai
Published: 2026-10-18

> A technical breakdown of LLM context windows, exploring physical KV cache memory bottlenecks, max generation ceilings, and frontier pricing across OpenAI, Anthropic, and Google.

A context window in an LLM is the hard ceiling on total tokens evaluated in a single inference pass, including input tokens, output tokens, and control tokens. It defines the absolute working memory available to the model during an API call rather than just the volume of text you can submit in your prompt. [1]

Understanding this distinction is critical for developers managing production budgets. While context sizes have scaled to 1M tokens across frontier architectures, physical hardware bottlenecks and asymmetric output limits change how multi-turn pipelines must be designed. [2]

## What a context window actually measures in an LLM

In transformer models, self-attention computes vectors of weights for each token in a sequence of text, where each weight represents how relevant that token is to others in the sequence. The size of the context window determines exactly how many token relationships can be tracked at one time. [3]

Because every request consumes part of this shared budget, developers tracking [Claude token pricing](/blog/claude-tokens-pricing-and-cost-guide) must account for the full scope of their calls. In documentation from OpenAI, the context window includes all input tokens, output tokens, and control tokens. When an LLM generates a response, each generated output token occupies space within that same context boundary alongside system instructions and conversation history. [1]

![Anthropic documentation table comparing Claude models, context windows, and output limits.](https://dygolixznokmekyvajbb.supabase.co/storage/v1/object/public/images/content/2f35334e53a99796f7f081e7.png)

## Total context window capacity versus max output tokens

A common mistake when designing agent pipelines is assuming that a model with an advertised context window can generate completions equal to that entire length. In practice, generation limits are often capped well below the total input boundary. [2]

For example, Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5 all provide a 1M-token context window with a max output limit of 128K tokens. By contrast, Google built [Gemini 4 Argon architecture](/blog/gemini-4-argon-architecture-benchmarks-pricing) to support long-decode generation up to 1M tokens, enabling generation sequences that match the entire breadth of its context capacity. [2][4]

Maximum output limits across frontier LLM architectures, in tokens. [2][4][10]

- Claude Opus 5.5: 128000 tokens
- Claude Sonnet 5.5: 128000 tokens
- Claude Haiku 5.5: 128000 tokens
- GPT-6 Astra: 128000 tokens
- Gemini 4 Argon: 1000000 tokens

## Hardware bottlenecks and attention degradation inside long windows

Expanding context capacity is bounded by physical computing constraints. Three architectural and hardware constraints limit context window sizes: transformers, GPU memory, and KV cache memory growth. [3]

During inference, GPUs move data around in fast memory (SRAM) with limited capacity and slow memory (HBM) with a higher capacity. Attention to tokens constantly shuffles between the two types, and the bottleneck on the GPU is its memory bandwidth. As conversation depth grows, storing the key-value states of prior tokens in the KV cache requires massive memory throughput. [3]

Beyond memory bandwidth, long contexts suffer from retrieval failure points. Language model performance can degrade significantly when accessing relevant information placed in the middle of long contexts compared to the beginning or end. This 'lost in the middle' phenomenon means that feeding an entire documentation corpus into an open prompt often underperforms compared to structured retrieval pipelines. [5]

## Comparing context limits and API costs across frontier models

Operating inside multi-hundred-thousand-token windows carries stark cost trade-offs. Standard pricing scales steeply on high-reasoning tiers, though lighter models provide massive margins for ingestion. [2][4][6][7]

Context window specifications and base pricing across frontier LLMs. [2][4][6][7][9][10]

| Model | Context Window | Max Output | Input / 1M Tokens | Output / 1M Tokens |
| --- | --- | --- | --- | --- |
| Claude Opus 5.5 | 1M tokens | 128K tokens | $4 | $20 |
| Claude Sonnet 5.5 | 1M tokens | 128K tokens | $2 | $10 |
| Claude Fable 5.1 | 1M tokens | 128K tokens | $10 | $50 |
| Claude Haiku 5.5 (<=100k tokens) | 1M tokens | 128K tokens | $0.10 | $0.50 |
| GPT-6 Astra (<272K context) | 1,050,000 tokens | 128,000 tokens | $10.00 | $50.00 |
| Gemini 4 Argon (through Nov 30, 2026) | 1M tokens | 1M tokens (long-decode) | $2.00 | $10.00 |

Standard API input pricing per 1M tokens across frontier models, in USD. [2][4][6]

- Claude Haiku 5.5 (<=100k): 0.1 $
- Claude Sonnet 5.5: 2 $
- Gemini 4 Argon: 2 $
- Claude Opus 5.5: 4 $
- Claude Fable 5.1: 10 $
- GPT-6 Astra (<272K): 10 $

From our data: For 1 million input plus 1 million output tokens, Anthropic Claude Haiku 5.5 costs $0.60 and OpenAI gpt-6-astra costs $60, 100 times as much. Standard rates on earlier tiers also demonstrate this spread, with Claude Opus 5 priced at $5 input and $25 output per million tokens, while Claude Sonnet 5 matched Sonnet 5.5 at $2 input and $10 output. [7]

Combined cost of 1M input plus 1M output tokens, in USD. [7]

- Claude Haiku 5.5: 0.6 $
- OpenAI GPT-6 Astra: 60 $

## How prompt caching alters large-context economics

Because loading large prompt prefixes repeatedly overloads GPU memory bandwidth, providers use prompt caching to reuse calculated KV cache states across requests. Understanding cache mechanics directly impacts the bottom line of multi-agent workflows. [3][8]

Anthropic prompt caching provides a default 5-minute lifetime measured from the start of the request that writes or reads the cache entry. Pricing multipliers charge 1.25 times the base input price for 5-minute cache writes and 2 times base input for 1-hour writes. However, as detailed in our guide to [Claude prompt caching](/blog/claude-prompt-caching), cache hits and refreshes on Claude Opus 5.5 and Claude Sonnet 5.5 are charged at 0.05 times the base input token price. [8]

Competitors implement different billing structures. GPT-6 Astra standard processing rates for context lengths under 272K charge $1.00 per million cached input tokens against its $10.00 base input rate. Gemini 4 Argon standard paid pricing through November 30, 2026, charges $0.10 per million tokens for context caching, relying solely on implicit caching without support for explicit user writes. Meanwhile, Gemini 3.8 Flash charges an hourly context caching storage price of $0.50 per 1,000,000 tokens through December 31, 2026. [4][6]

![Google Developer API pricing sheet displaying standard input, output, and context caching rates for Gemini 4 Argon.](https://dygolixznokmekyvajbb.supabase.co/storage/v1/object/public/images/content/346ffc364ad14358b4269ac1.png)

Limit: Official provider documentation does not publish uniform memory bandwidth saturation figures or precise KV cache byte counts per layer across proprietary architectures. Furthermore, empirical middle-context attention drop-offs vary significantly depending on document formatting and task complexity.

When building applications requiring large contexts, decouple context capacity from output needs. If a workflow demands massive context reuse across multi-turn queries, select architectures supporting aggressive prompt caching discounts like Gemini 4 Argon or Claude Sonnet 5.5, and place mission-critical facts at the immediate beginning or end of your prompts to prevent retrieval degradation. [4][5][8]

## Sources

1. [What does "CONTEXT WINDOW" mean in the documentation - API - OpenAI Developer Community](https://community.openai.com/t/what-does-context-window-mean-in-the-documentation/566158)
2. [Models overview - Claude Platform Docs](https://docs.anthropic.com/en/docs/about-claude/models)
3. [LLM Context Windows for Dummies by a Dummie | by Dhru Patel | Medium](https://medium.com/@Scoper/llm-context-windows-for-dummies-by-a-dummie-13926135a02f)
4. [Gemini Developer API pricing  |  Gemini API  |  Google AI for Developers](https://ai.google.dev/pricing)
5. [[2307.03172] Lost in the Middle: How Language Models Use Long Contexts](https://arxiv.org/abs/2307.03172)
6. [Business Pricing | OpenAI](https://openai.com/api/pricing/)
7. [LLM API pricing timeline (3AM Marketer data set)](https://3ammarketer.com/data/model-pricing)
8. [Prompt caching - Claude Platform Docs](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching)
9. [Anthropic's Claude Haiku 5.5 Pricing: $0.10 per million input tokens | Simple Budget posted on the topic | LinkedIn](https://www.linkedin.com/posts/simplebudgetapp_androiddev-flutter-aiagents-activity-7514656272261124096-cr8a)
10. [GPT-6 Astra 1 Million Token Context Window Explained: Limits, Costs and Best Practices | Elser AI News](https://www.elser.ai/news/gpt-6-astra-one-million-token-context-window)