Working with Contexts – O’Reilly
September 16, 2025
Managing Your Context Is the Key to Successful Agents
As frontier model context windows continue to grow,1 with many supporting up to 1 million tokens, I see many excited discussions about how long-context windows will unlock the agents of our dreams. After all, with a large enough window, you can simply throw everything into a prompt you might need—tools, documents, instructions, and more—and let the model take care of the rest.
Long contexts kneecapped RAG enthusiasm (no need to find the best doc when you can fit it all in the prompt!), enabled MCP hype (connect to every tool and models can do any job!), and fueled enthusiasm for agents.2
But in reality, longer contexts do not generate better responses. Overloading your context can cause your agents and applications to fail in surprising ways. Contexts can become poisoned, distracting, confusing, or conflicting. This is especially problematic for agents, which rely on context to gather information, synthesize findings, and coordinate actions.
Let’s run through the ways contexts can get out of hand, then review methods to mitigate or entirely avoid context fails.
Source: Working with Contexts – O’Reilly
Certainly, when it comes to software development using large language models, the context we provide the model seems to have a very significant impact on the model’s performance.
We spend a lot of time thinking about what makes for a good prompt, but less so thinking about how we manage the contexts we provide our models.
