What is just-in-time context?
Just-in-time context is a context-engineering approach in which an AI agent holds lightweight references, such as file paths, stored queries or links, and loads the underlying data with tools only at the step that needs it.
The older pattern for giving a model knowledge is to retrieve everything that looks relevant before the model starts, then paste it into the prompt. Just-in-time context inverts that. The agent starts with pointers and decides, step by step, which ones to open.
Anthropic's "Effective context engineering for AI agents" (2025-09-29) describes the approach as mirroring how people work: nobody memorises a whole codebase, they keep a file system and bookmarks and look things up when needed.
How it works
References instead of payloads
The context holds identifiers: file paths, saved queries, URLs, record IDs. Tools turn an identifier into content when the agent asks.
Partial reads
Anthropic's example is Claude Code running targeted queries and using commands such as head and tail to inspect large data without loading the whole object into context.
Metadata as signal
Folder structure, naming and timestamps tell the agent what a file is likely to hold before it opens it. A test helper in a tests folder implies a different purpose from one in core logic.
Handles that expand and contract
Google's context-aware framework post (2025-12-04) keeps large objects as artifacts the agent sees only by name and summary. A load tool brings the content in for one call, and it is offloaded again afterwards by default.
The hybrid most teams end up with
Pure runtime exploration is rarely the whole answer. Anthropic describes a hybrid in which some context is loaded up front for speed and the rest is explored as needed. Claude Code loads its project memory file at the start, then finds code with glob and grep as it goes. Anthropic suggests the up-front share can be larger in less dynamic domains such as legal or finance work.
Anthropic's memory tool documentation frames memory the same way: the agent records what it learns in files and reads them back on demand, instead of carrying everything in the active context.
Where it fails
Slower than precomputed retrieval
Each lookup is a tool call and a model turn. Anthropic notes runtime exploration is slower than retrieving precomputed data.
Dead ends
Without guidance on where to look, an agent can spend its context on searches that go nowhere or misuse tools.
Weak references
A handle with a vague name or summary never gets opened. The quality of the index decides what the agent finds.
Just-in-time context, RAG and progressive disclosure
Retrieval-augmented generation decides what the model sees before it starts, usually by similarity search. Just-in-time context lets the agent decide during the run. Progressive disclosure is the same idea applied to capabilities, such as skills and tool definitions, where just-in-time context usually refers to data: files, records and documents.
How Motiontic's engineers apply it
Motiontic's engineers give agents tools that return pages, summaries and IDs by default and full content on request, and write tool descriptions that say where things live. Evals check both answer quality and the number of lookups, so a slower path is accepted only when it buys accuracy.
Up-front retrieval compared with just-in-time context
| Up-front retrieval | Just-in-time context | |
|---|---|---|
| Who decides what loads | The pipeline, before the model runs | The agent, step by step |
| What sits in context | Retrieved passages | References, metadata and what has been opened |
| Speed | One retrieval, one call | More tool calls and turns |
| Freshness | Depends on the index | Reads the live source |
| Main risk | Irrelevant passages crowd the prompt | Dead-end exploration or a missed reference |
Common questions
What is just-in-time context in AI agents?
It is a pattern where the agent keeps references, such as file paths or query IDs, in its context and uses tools to load the actual content only when a step needs it.
Is just-in-time context better than RAG?
Neither wins everywhere. Up-front retrieval is faster for simple question answering over a stable corpus. Just-in-time context suits agents that work across live files and systems, at the cost of more tool calls.
How is just-in-time context different from progressive disclosure?
They share the idea of loading on demand. Progressive disclosure is usually applied to skills and tool definitions; just-in-time context usually refers to data such as files, records and documents.
Related
Progressive disclosure
Show the menu first, load the dish only when it is ordered.
RAG vs Agentic Search
One retrieval before the answer, compared with an agent that searches as it works.
Agentic search
Retrieval as a tool the agent calls, step by step, rather than a pipeline stage.
Artifacts (AI agents)
Large data kept out of the prompt, or outputs that show a person what the agent did.
AI agent development
Agents that do real work for users, with the evals, traces and guardrails to keep them in production.
Running agents that have to hold up?
Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.