An agent works for two hours. The harness compacts the transcript, the run continues, and somewhere in the next twenty turns it re-runs a migration it has already run, asks the user for a value the user typed an hour ago, or refactors a file it was explicitly told not to touch. Nothing in the run log marks the turn where that fact disappeared. The summary reads fluently, the task still finishes, and the only trace of the loss is work the agent repeats.

That is the reader problem this post takes up: which pinned facts survive a summarisation pass, what the pass costs in tokens, and how to make the facts that cannot be lost survive it. The framing matters, because “which facts survive” is an empirical question and the honest answer is narrower than the advice literature implies. Every number below is quoted from a published study or a vendor document and linked where it appears. No retention figure here is a measurement performed by this article; what the post adds is a reading of the published evidence and the shape of a retention test an operator can run against their own harness.

Three shipped compaction paths, and they lose different things

Compaction is not one mechanism. Anthropic’s production API path replaces older conversation turns with a summary that the model writes on the server, so the client needs no summarisation code of its own (https://platform.claude.com/docs/en/build-with-claude/compaction). Because the summary is generated rather than selected, its content is not deterministic across runs or model versions. The API summariser’s own prompt and target length are unpublished, and no per-fact measurement of what that summary retains is published either — which is a narrower statement than vendor-wide silence on retention, as the next section shows.

The second path is deletion rather than summarisation. The context-editing API’s clear_tool_uses_20250919 strategy “automatically clears the oldest tool results in chronological order” and replaces each cleared result with placeholder text signalling removal (https://docs.claude.com/en/docs/build-with-claude/context-editing). Chronological age is the only criterion, so “oldest” is not “least relevant”: the file listing from turn 3 goes before the wrong hypothesis from turn 30, and a search result that established a fact can vanish while the conclusion drawn from it stays.

The third path is the client harness. Claude Code auto-compacts when a conversation approaches its context limit, accepts per-run preservation instructions through /compact, and treats the resulting cache miss as an expected rebuild, because the rewritten conversation “invalidates the conversation layer: the next request has a new, shorter history that doesn’t share a prefix with the old one” (https://code.claude.com/docs/en/prompt-caching). For this harness the vendor also publishes a mechanism-level survival table: CLAUDE.md and auto memory are re-injected, up to five files are re-read, and skill bodies are re-injected up to 5,000 tokens each with the oldest dropped first (https://code.claude.com/docs/en/context-window#what-survives-compaction). That is not a per-fact retention rate, and it says nothing about the API summariser, but it is the closest thing to a published answer for any single implementation — and it makes the escape hatch concrete. Preservation instructions are a supported input, which makes “pin these facts” a documented affordance rather than a hack.

All three share one property that matters operationally: the compaction event is a rewrite of prior context, not an append. Whatever the pass drops is gone from the window rather than demoted within it.

A passing summary is not evidence of fidelity

The intuitive test — read the summary and see whether it looks right — fails because fluency is exactly what a summariser optimises. Anthropic’s own guidance concedes the difficulty: “the art of compaction lies in the selection of what to keep versus what to discard, as overly aggressive compaction can result in the loss of subtle but critical context whose importance only becomes apparent later” (https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents). The same guidance recommends tuning for recall before precision: a summary that carries a fact you did not need costs tokens, while a summary that omits one costs a wrong action.

Measured work on the failure mode agrees that completion is the wrong dependent variable. In a deterministic planning environment with a fixed 24-turn horizon, 5x compression moved GPT-5.5 completion from 80% to 85% — statistically nothing, p = 1.0 — while retrieval calls rose from 21.0 to 63.9 at p = .002 (https://arxiv.org/abs/2608.16370). Retrieval increased in all six model-regime comparisons, and completion changed significantly only for DeepSeek at 10x compression. The compressed agent finished as often and paid for it in reacquisition: it went back for state the uncompressed agent still had in front of it. The authors also report that the signature is environment-dependent, with ALFWorld showing no such retrieval surge, which is a useful warning against generalising any single ratio.

A second line of work catalogues the qualitative shape of that cost. A preliminary study of verifier-guided compaction reports that compression “can weaken the influence of recent interactions, increasing blocked actions, repeated exploration, and instability across runs” (https://arxiv.org/abs/2608.06503). The authors label it preliminary and report no per-fact retention rate, so it supports a taxonomy rather than a number: the symptoms are blocked action, repeat exploration and run-to-run variance, and all three are invisible in a completion-rate dashboard.

Which fact classes fail first

The sharpest per-fact evidence comes from a controlled gist-compression study over all ten LoCoMo conversations at temperature 0 (https://arxiv.org/abs/2608.11775). Its question counts and its preservation numbers do not share a denominator, and the paper is explicit about which is which. The study evaluates 1,935 matched text-only questions across the ten conversations, and its primary accuracy aggregate uses the 1,501 that remain once Category 5 (adversarial) questions are excluded. The preservation figures are a separate analysis: Table 3 reports verbatim preservation rates across all ten conversations over the 952 temporal expressions it found, so 3.05% before a one-sentence prompt fix and 62.39% after it are expression-level rates, not question-level accuracies. Named-entity and event preservation rates “barely change (x1.02 and x1.11)”. What the fix moved at question level is judge accuracy on category-2 (temporal) questions: +0.314 [0.254, 0.375], paired over n = 315 questions in the matched set. Both readings describe the same mechanism — the gist abstraction kept who did what to whom and dropped when.

Two things follow. First, the failure is class-specific, not global: the authors trace it to a mechanism — the gist abstraction preserved relational and event structure while omitting when things happened — and they report preservation per content type rather than as one blended rate. A coding agent’s most-reused classes are the ones this study finds fragile on the temporal side — file paths, command strings, version numbers, the exact revision a decision was made against — because those are identifier-shaped values that a prose summary naturally paraphrases into something lossy rather than wrong-looking. Second, the entire difference came from wording: the paper’s improved condition is “identical to SWC-Full except for one added sentence at the prompt start”, and that one sentence took expression-level temporal preservation from 3.05% to 62.39% with no other change to the pipeline. The lever is instruction, not architecture.

Why class matters at all is the older result on position. Performance is “often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts” (https://arxiv.org/abs/2307.03172). Compaction changes where a surviving fact sits: a constraint from the opening turns survives only if the summary places it somewhere the model still attends to, and the middle of a reconstructed summary is the worst place for it. Chroma’s Context Rot report extends the same caution across a modern panel, finding that performance varies significantly with input length even on simple tasks across the 18 models evaluated (https://research.trychroma.com/context-rot) — so a longer, more faithful summary is not automatically a safer one.

The control arm you cannot skip

Any retention test that omits a no-compaction control measures the wrong thing. If the same probes fail on the full transcript, the loss belongs to the model’s recall over 10k tokens, not to the summary, and no compaction fix will help. Two decision rules follow from that, and both are falsifiers rather than heuristics.

If the control arm fails the probes, fix retrieval or shorten the task before touching compaction. If all three strategies retain 95% or more of the pinned facts at 60% or better token reduction, the silent-loss premise is false for your harness and the correct change is to delete the mitigation. A retention number without a matched control is not a measurement of compaction.

A retention test for your own transcripts

The protocol below is a runnable specification, not a result. It is written so that a harness engineer can implement it in an afternoon against their own agent, and it deliberately reuses the design choices of the studies above rather than inventing a scoring scheme.

  1. Build five transcripts of 8,000 to 12,000 tokens each from real runs, and seed each with 12 pinned facts spread across four classes: user constraints, file paths, decisions, and rejected approaches. Add six plausible distractors of the same shape, so a correct answer cannot be reached by picking the only concrete-sounding string in the window.
  2. Write one probe per pinned fact. Each probe must require the exact value rather than a description of it, so scoring can be exact match or substring match against the seeded string. Score five outcomes, not two: correct, paraphrase that preserves the value, fabrication of a plausible value, explicit abstention, and silence.
  3. Run three strategies on each transcript: truncation of the oldest window, a free-text summary produced by the harness’s own summariser, and a structured summary with an explicit pinned-facts block carried verbatim outside the summarised span.
  4. Ask the probes through the same model, at the same settings, on the compacted context and on the full transcript. The full-transcript arm is the control described above and is not optional.
  5. Repeat each cell three times and report the spread rather than the mean, because summary generation is stochastic and a single run hides the variance that matters to an operator.
  6. Count tokens before and after with the server’s own tokenizer, and record the model id, quantisation, harness version, sampling settings, transcript hash and the exact compaction prompt. Without those fields the retention rate is unreproducible and should not be quoted.

The template and the scorer ship with this page. probe-template.json is the fixture schema: transcript metadata (model id, quantisation, harness version, compaction prompt, token counts, tokenizer), the 12 seeded facts with their class and the exact string that counts as retained, six same-shape distractors, and one probe per fact. score_retention.py reads a responses file and applies the five outcome rules above mechanically — substring match after case and separator normalisation, distractor match before keyword match, abstention cues last — then reports retention and exact-match rates per strategy, the per-class counts, the token reduction from the two recorded counts, and whether a control arm is present. It needs only the Python standard library and no API key. test_score_retention.py runs the scorer against a hand-written synthetic fixture — the two data files ship beside it as fixtures/fixture-probes.json and fixtures/fixture-responses.json — and asserts the expected counts; six tests passed on Python 3.12.3 at 2026-11-02T00:50:15Z, with the captured run in fixture-test-output.txt. That fixture tests the scoring rules, not a model: no transcript was compacted to produce it, and it is not a retention result. The protocol is still yours to run — the files remove the writing, not the measurement.

The fabrication column is the one to watch. A summariser that drops a path but lets the model guess a plausible sibling path produces a confident wrong action, which is worse than an abstention and much harder to detect after the fact. The published taxonomy predicts blocked actions and repeated exploration (https://arxiv.org/abs/2608.06503); fabrication is the same loss with better acting.

What to pin, and where to pin it

The studies support three practical moves. Carry a pinned-facts block verbatim: constraints, absolute file paths, accepted decisions and rejected approaches, each as one line that no summariser is allowed to rewrite. Give the summariser an explicit preservation instruction covering value classes rather than topics — dates, versions, paths, numbers — since the one-sentence instruction in the LoCoMo study moved temporal preservation from 3.05% to 62.39% (https://arxiv.org/abs/2608.11775). And log the diff, not just the event: record which pinned facts appear in the summary before and after the pass, so that a later re-run or re-ask can be attributed to a specific compaction rather than to model drift.

For the surrounding strategy, the corpus has adjacent material rather than duplicates: the sixty-percent threshold rule and its premature-compaction pitfall are covered in Claude Code context management, the survey of compaction and memory strategies in agentic workflow context management, the cross-tool case in portable handoff context, and compaction as a harness design decision in harness design for coding agents. None of them reports a retention rate, which is the gap this page is built around.

Limits

Compaction is harness-specific. Each number above belongs to one implementation: a vendor API summary path, one client harness, or one research framework, and none of them transfers as a universal retention rate. The per-fact figures come from LoCoMo conversational memory at temperature 0 with judge scoring, not from coding-agent transcripts, and file paths enter those results only through a named-entity proxy. The reacquisition result comes from a synthetic bounded-horizon environment whose own ALFWorld arm did not reproduce the signature, and the instability study is labelled preliminary by its authors. Chroma’s panel is vendor research, and its model versions predate this page. Read the numbers as evidence about which classes are fragile and which measurement design exposes them, not as a benchmark of any model’s long-context quality.