Compaction is the technique an AI assistant uses to compress an over-long conversation into a shorter summary so that work can continue inside a fixed context window. A technical explanation of one tool’s approach drew attention on Hacker News.
Key takeaways
- Compaction is the process of replacing a long conversation history with a condensed summary so an AI assistant can keep working within a fixed context limit.
- Every large language model has a bounded context window, and long agent sessions reach that bound quickly because tool output and file contents are verbose.
- Most compaction schemes summarise older turns while preserving the recent messages, the stated goal and any pending state more or less verbatim.
- Compaction is lossy by design, so the central engineering question is not how to compress but which details are safe to discard.
- Users can reduce their reliance on compaction by keeping durable information in files and notes rather than in the conversation itself.
What is actually being described
The item circulating is an explanation of how a particular AI assistant handles compaction: the moment when a running session has accumulated more conversation than the underlying model can accept, and something has to be removed. The specific implementation details of that tool are not restated here, because they cannot be independently verified from the trend signal alone. What can be described is the general mechanism, which is broadly similar across the assistants and coding agents now in common use.
A language model does not have memory in the ordinary sense. It has an input, and everything the assistant knows about the current session — the instructions, the user’s messages, the files it has read, the commands it has run and their output — must be packed into that input each time it is called. That input has a maximum size, usually called the context window, measured in tokens.
When a session approaches the limit, the assistant has three options: fail, drop the oldest material, or rewrite the history into something shorter. Compaction is the third option. The assistant asks a model to produce a structured summary of what has happened so far, discards the raw history, and continues from the summary.
Why it is being discussed now
Interest in compaction has grown alongside the shift from single-question chatbots to agents that run for long stretches without supervision. A chat exchange rarely exhausts a modern context window. An agent that reads a dozen source files, runs a test suite, reads the failure output, edits code and repeats will exhaust it routinely, sometimes several times in a single task.
That change moves compaction from an obscure implementation detail to something users notice directly. When it works, a long session simply keeps going. When it works badly, the assistant appears to forget a constraint it was given an hour earlier, repeats work it has already completed, or reverts a decision the user thought was settled. Because the failure is silent — nothing errors, the model simply no longer knows something — it is easy to mistake for a general lapse in capability.
Detailed write-ups of how a specific system handles the problem therefore attract a technical audience. They are, in effect, documentation of where a tool will lose the thread, which is useful information for anyone who depends on it.
The background a newcomer needs
Text handed to a language model is broken into tokens, roughly fragments of words. The context window is the maximum number of tokens the model can process at once, and it covers both the conversation so far and the reply being generated. Windows have grown substantially across successive model generations, but they remain finite, and the cost and latency of a request generally rise with the amount of context supplied.
Two consequences follow. First, an unbounded conversation is not possible, regardless of how large the window becomes. Second, filling a window is not free even when it is technically possible, so systems have an incentive to keep working context small rather than merely under the limit.
The word compaction is borrowed from storage engineering, where log-structured systems periodically merge and rewrite accumulated files to reclaim space and remove superseded data. The analogy is imperfect but instructive: in a database, compaction discards records that have been overwritten, and the result is expected to be equivalent to what came before. In a conversation, the system must judge what has been superseded, and that judgement can be wrong.
Who is affected and how
The most exposed group is developers using agentic coding tools, because their sessions are long, their tool output is bulky and the material at risk of being summarised away — a specific file path, a particular error string, an instruction not to touch a given directory — is often precise rather than general. Summarisation preserves gist well and preserves exact strings poorly.
Anyone building on top of a model API faces the same problem one level down, and must decide for themselves when to compact, what to keep and how to signal to the model that earlier detail has been condensed. There is no single accepted answer, and implementations vary in how aggressive they are.
Ordinary users of consumer assistants are affected more indirectly. Long-running conversations and persistent memory features rest on some form of the same compression, and the perceived quality of a system’s memory is partly a function of how well it decides what to retain.
Cost is a further consideration. Compaction reduces the tokens carried into each subsequent request, so it can lower the running cost of a long session, while the summarisation step itself consumes a model call.
Where informed people disagree
The first disagreement concerns whether compaction is a stopgap. One view holds that as context windows grow and the cost of long inputs falls, aggressive summarisation will matter less and systems should simply carry more history. The opposing view is that agents generate context faster than windows grow, that model attention over very long inputs is uneven, and that deliberate curation produces better results than dumping everything in.
The second concerns who should decide. Automatic compaction is convenient and can be triggered without interrupting the user; manual or user-prompted compaction is more predictable and lets the person confirm what is being preserved. Systems differ in how much control they expose.
The third concerns method. Summarising with the model is flexible but lossy and non-deterministic. Alternatives include dropping the middle of a conversation while keeping the beginning and end, offloading detail to files or an external store and retrieving it on demand, or maintaining a structured record of task state that is updated rather than summarised. These approaches are often combined, and there is no consensus on the right mix.
What this means in practice
The practical implication for users is that anything which must survive a long session should not live only in the conversation. Writing requirements, constraints and decisions into a file — a task list, a notes document, a project instructions file — means they can be re-read after compaction rather than reconstructed from a summary.
It also helps to keep sessions scoped. Starting a fresh session for a distinct task avoids carrying irrelevant history, and reduces the chance that a summary compresses two unrelated threads into one confused account. Where a tool exposes a manual compaction command, using it at a natural boundary is more reliable than letting it fire mid-task.
For builders, the useful framing is that compaction is a retention policy, not a compression algorithm. Deciding in advance which categories of information are load-bearing — the goal, the constraints, the current state, the things already tried and rejected — and preserving those structurally tends to work better than asking a model to summarise an undifferentiated transcript. It is also worth making compaction visible, so that a user can tell when the assistant’s view of the session has been rewritten.
What to watch next
Three developments are worth following. The first is whether context windows and long-context reliability improve enough to make routine compaction unnecessary for typical sessions, or whether agent workloads continue to outpace them. The second is the emergence of shared conventions: at present each system handles retention differently, and any convergence on a common approach to session state would make behaviour more predictable across tools.
The third is transparency. Because compaction determines what an assistant can still be held to, clearer signalling of when it has occurred, and what was kept, would let users work with the limitation rather than around it. Whether vendors document this consistently remains to be seen.
Frequently asked questions
What is compaction in an AI assistant?
Compaction is the process of shortening a conversation that has grown too long for the model’s context window. Rather than failing or silently dropping the oldest messages, the assistant produces a condensed summary of the session so far, discards the raw history, and continues from that summary. It is a form of managed forgetting that lets long sessions keep running within a fixed input limit.
How is compaction different from just having a longer context window?
A longer window raises the ceiling; compaction manages what happens when the ceiling is reached. Larger windows reduce how often compaction is needed, but they do not remove the need, because agent sessions can generate history indefinitely. Longer inputs also cost more and take longer to process, so keeping working context small has value even when a bigger window is available.
Does compaction lose information?
Yes, by design. Summarising a conversation necessarily discards detail, and the detail lost is often precise material such as exact file paths, error strings or specific instructions given early in a session. Well-designed systems try to preserve the task goal, active constraints and current state more or less verbatim while compressing the narrative around them, but no scheme is lossless.
Why do coding agents run out of context so quickly?
Because their inputs are dominated by machine-generated text rather than human conversation. Reading source files, listing directories, running builds and capturing test output all produce large volumes of tokens, and each is added to the session history. A handful of file reads and a verbose test failure can consume more context than a lengthy human dialogue, so limits arrive far sooner than users expect.
Can I control when compaction happens?
It depends on the tool. Some assistants compact automatically as the limit approaches, some expose a command to trigger it deliberately, and some do both. Where manual control exists, invoking it at a natural boundary between tasks is generally more reliable than letting it happen partway through a piece of work, because the summary is then produced from a coherent unit.
Is this the same as compaction in databases?
The name is borrowed from storage engineering, where compaction merges and rewrites accumulated data files to reclaim space and remove superseded records. The intent is analogous, but the guarantees differ sharply. Database compaction is expected to preserve the meaningful state exactly, whereas conversational compaction relies on a model’s judgement about what matters and can therefore discard something that later turns out to be important.
Sources and further reading
- Hacker News — the discussion thread in which the original technical explanation was shared and debated by practitioners.
- Published developer documentation from AI assistant and agent vendors, which describes context limits and session management behaviour.
- Peer-reviewed and preprint literature on long-context language models, attention over long inputs and automatic summarisation.
- Database and storage systems literature on log-structured merge trees, the origin of the term compaction.
Surfaced from the hackernews signal “AI context compaction explainer”. AI-assisted draft, editorially reviewed.
