What Is Context Offloading?

What is context offloading?

Drew Breunig's How to Fix Your Context (June 2025) lists offloading as one of six tactics alongside RAG, tool loadout, context quarantine, context pruning and context summarization. He defined it as "the act of storing information outside the LLM's context, usually via a tool that stores and manages the data." His example was Anthropic's "think" tool, which he described as "basically a scratchpad."

LangChain's Context Engineering post (July 2025) groups agent context strategies into four buckets: write, select, compress and isolate. Offloading corresponds to the "write" bucket, though the post does not use the word: "Writing context means saving it outside the context window to help an agent perform a task." It separates scratchpads, which last for one task, from memories, which persist across sessions; "select" is the return trip.

The idea predates the name. MemGPT (Packer et al., 2023) proposed "virtual context management," modeled on operating systems that move data "between fast and slow memory." Anthropic's guide to context engineering calls the note-writing form "structured note-taking, or agentic memory."

How context offloading works

Two forms are common. The agent writes its own notes, such as a plan file or progress log it re-reads after the window is compacted or reset. Or the harness offloads bulky tool output. LangChain's Deep Agents SDK is a documented example: when a tool response exceeds 20,000 tokens, it writes the response to the filesystem and substitutes "a file path reference and a preview of the first 10 lines." As the session crosses 85% of the model's window, it also truncates older write and edit tool calls, replacing them with pointers to the files on disk (Curme and Daugherty, LangChain, 2026).

A minimal harness hook, illustrative:

def on_tool_result(step, result, window, store, threshold=20_000):
    if count_tokens(result) <= threshold:
        window.append(result)
        return
    path = store.write(f"tool_results/step_{step}.txt", result)
    window.append(f"[Full output saved to {path}. First lines:]\n{head(result, 10)}")
    # The agent can read or grep the file later and pull back only what it needs.

Unlike summarization, offloading is lossless: the full content can be read back. The cost is retrieval. The agent has to know what was offloaded, where it went and when to fetch it, and LangChain's Context Engineering post notes that memory selection "is challenging."

Why context offloading matters in real-time systems

An offloaded note is a copy of what was true when the agent wrote it. That is harmless for content that does not change, such as a search result, a document or the agent's own plan. It is a hazard for business state. A note that records an account balance, a remaining budget or an inventory count starts aging the moment it is written, and re-reading it later returns the past with no sign that it is old. That is context lag, created by the agent's own storage, and no prompt technique repairs it.

The practical rule is to offload references to mutable facts, not copies of them: the account ID and the query, not the balance. Anything a decision gates on should be read from its source when the decision runs. When several agents write to the same offloaded workspace, it stops being a scratchpad and becomes shared mutable state. Concurrent writes then need the concurrency control of a real store, not last-writer-wins on a file. The difference between a fact degrading inside the window and a fact that was stale before it got there is covered in context rot vs. context lag.

FAQ