Long Context Becomes Standard, 1M+ Tokens Everywhere

What Is Happening

Million-plus token context windows are now standard on flagship models. Gemini 3.1 Pro ships a 2M-token window. Anthropic’s Claude Opus 4.7, released April 16, 2026, ships 1M tokens for API customers. Llama 4 Scout, released April 5, 2026, ships a 10M-token context window in open weights.

At 1M tokens, a single prompt can hold an entire mid-sized codebase, a full book, a year of meeting transcripts, or a complete product spec plus customer research. At 10M tokens, it can hold all of the above combined.

The practical effect is that retrieval-augmented generation, the dominant 2023-2024 architecture pattern, has become optional for many use cases. Instead of chunking documents into a vector database and retrieving the top-K chunks per query, teams load the whole document into context and let the model attend across all of it.

Why It Matters

For developers: The RAG-first default is broken. For many apps, especially internal tools, research assistants, and document Q&A, dumping the whole corpus into context produces better answers than chunked retrieval and costs less engineering.

For product quality: Models that actually use all of their context window (rather than just claiming it) handle follow-up questions, cross-document reasoning, and multi-file code edits without the retrieval layer dropping critical context.

For infra spend: Vector DB licenses, embedding. Pinecone, Weaviate, and Chroma shift from “required infra” to “required for specific use cases.”

Catch: Context window size and context window quality are not the same. Needle-in-a-haystack tests still show degradation in the middle of very long windows on most models. “Effective context” is often 30-60% of advertised context for complex reasoning.

Who Is Winning

Google Gemini. 2M tokens, well-utilized, with native multimodal input (PDFs, video, audio) has made Gemini the default for document-heavy workflows.

Anthropic Claude. Claude Opus 4.7 at 1M tokens with strong attention over long context is the choice for code and legal work, and is the default engine inside Claude Code.

OpenAI GPT-5.5. Caught up to the 1M frontier in April 2026 and is integrated across ChatGPT Enterprise and the API.

Meta Llama 4 Scout. 10M tokens in an open-weight model changes the ceiling for self-hosted applications. Mistral Large 3 at 512K under Apache 2.0 broadens that choice for European and sovereign buyers.

Losers: Pure RAG tooling companies that built moats around vector retrieval are pivoting to hybrid or getting squeezed. NotebookLM, which treats the uploaded corpus as the context, is the first-party example of the new pattern.

What To Watch Next

Attention quality over window size. The next race is not “larger window” but “better attention inside the window.” Expect benchmarks focused on middle-of-context retrieval, cross-document reasoning, and long-horizon planning. ICLR 2026, which opens in Rio on April 23, 2026, is showcasing several long-context attention papers that will set the next year of evaluation language.

Caching strategies. Prompt caching (Anthropic shipped this in 2024, OpenAI and Google followed) changes the economics of long-context workloads. Loading 500K tokens once and asking 50 questions against it is now viable.

Agent memory. Long context is not the same as persistent memory. Agents that need state across sessions still need a memory layer. See the AI Memory Layer trend.

Security and prompt injection. Longer windows mean more surface area for adversarial input. Expect injection attacks and sandboxing frameworks to become a recurring topic.

How This Affects You

Builders: Before reaching for a vector DB, ask: does this fit in a 1M context? For many tools, yes, and the result is simpler and cheaper.

Users: Upload full reports, codebases, or archives instead of pasting excerpts. Modern flagships handle it.

RAG vendors: Position against hybrid use cases (very large corpora, frequently updated content, strict access controls). Pure “vector search for LLMs” is no longer enough.