What Is Context Compression?
Context compression is the practice of reducing the number of tokens in a language model's context while keeping the information the current task needs. Common forms are summarizing earlier turns, pruning irrelevant passages, trimming old messages by rule and token-level prompt compression. It is a technique, not a failure: it trades fidelity for room and focus, and an aggressive compressor can drop a detail the task later depends on.
Updated
What is context compression?
In agent engineering, compression is one of four context strategies in LangChain's Context Engineering for Agents (July 2025), alongside write, select and isolate: "Compressing context involves retaining only the tokens required to perform a task." The post describes two forms, summarization and trimming. Drew Breunig's How to Fix Your Context covers the same ground with two of its six tactics: context pruning, "removing irrelevant or otherwise unneeded information from the context," and context summarization, "boiling down an accrued context into a condensed summary."
Anthropic calls the summarize-and-restart form compaction: "the practice of taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary" (Effective context engineering for AI agents, September 2025). In Claude Code, the post says, the model "preserves architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs or messages." The post calls tool result clearing, dropping raw tool output the agent no longer needs to see, "one of the safest lightest touch forms of compaction."
Research also studies prompt compression as a model problem. LLMLingua uses a small language model to drop tokens that carry little information, and reports "up to 20x compression with little performance loss" on its benchmarks. Gist tokens train a model to compress a prompt into a smaller set of learned tokens, with "up to 26x compression of prompts." Provence prunes retrieved passages for question answering "with negligible to no drop in performance."
How context compression works
Each method removes something different and fails differently:
| Method | What it removes | Done by | Main risk |
|---|---|---|---|
| Trimming | Oldest messages or tool results, by rule | Application code | Drops an early constraint |
| Pruning | Passages irrelevant to the question | The LLM or a trained pruner | Cuts a passage that mattered |
| Summarization or compaction | Detail, keeping decisions and open items | The LLM itself | Loses context whose value shows up later |
| Token-level compression | Low-information tokens | A small language model | Output is hard for people to audit |
| Learned compression | Text, replaced by learned tokens | A model trained for it | Requires model training |
Compression is not only about fitting the window. Breunig notes benefits "beyond staying within the total context limit," because a growing context "becomes distracting and causes the model to rely less on what it learned during training," the failure he calls context distraction.
Two practices make compression safer. First, keep context structured. Breunig argues for "maintaining a structured version of your context in a dictionary or other form, from which you assemble a compiled string prior to every LLM call," so instructions and goals survive while history and documents are pruned or summarized. Second, tune the compactor against real traces. Anthropic warns that "overly aggressive compaction can result in the loss of subtle but critical context whose importance only becomes apparent later," and recommends starting by "maximizing recall" before working on precision.
Why context compression matters in real-time systems
A summary is a snapshot of the context at the moment it was written. Any live value it carries, such as a balance, a stock count or an order status, is as old as the summary, and nothing in the compressed text says so. An agent that compacts on turn 40 and acts on turn 90 can act on a number that was right fifty turns earlier. The compressor created that staleness inside the prompt.
That is separate from context lag, where the value was already behind its source when it was retrieved. Compression cannot repair a lagging read, and it can make one look more settled by restating it as a conclusion. The practical rule: compress reasoning, decisions and constraints, not live values. Keep a reference, such as an account ID, and re-read the value at the step that acts on it. How in-prompt fixes differ from fixes to the data underneath is covered in context rot vs. context lag.
FAQ
Related terms
A token budget limits how many tokens an LLM call or agent task may use: how the context window is split, how much output and reasoning, and total spend.
Context offloading moves information out of an LLM's context window into external storage, such as files or notes, and pulls it back only when a step needs it.
Context isolation splits an agent's work across separate context windows, such as sub-agents, so each model call sees only what its part of the task needs.
Context distraction is when a context grows so long that a language model leans on it, such as its own past actions, over what it learned in training.
Context poisoning is when a hallucination or other error enters a language model's context and is referenced again on later turns, so the mistake compounds.
Context lag is the delay between a fact changing at its source and the system an automated decision reads from reflecting it. Also called stale context.