Apple's SELFCOMPACT lets models decide when to compress agent traces

Apple researchers introduced SELFCOMPACT, a scaffold that lets a language model decide when and how to compact its own agent trace, instead of relying on fixed-interval triggers. The work, published for NeurIPS, targets long chains of thought and tool calls that accumulate stale content, anchor later generations, and eventually outgrow the context window.

Existing scaffolds compact at a token threshold, which ignores trajectory structure and risks discarding partial results mid-derivation or mid-search. SELFCOMPACT pairs two inference-time elements: a compaction tool the model invokes to summarize accumulated context, and a lightweight rubric specifying when to fire, such as when a sub-task has resolved or the trajectory is converging, and when to suppress, such as mid-derivation or when stuck. The tool alone is unevenly used across open-weight models, often invoked at unhelpful moments or not at all; the rubric alone cannot act. Together, they elicit adaptive compaction without fine-tuning or external supervision.

Results across six benchmarks and seven models

The paper reports empirical results on six benchmarks covering competitive math and agentic search, across seven models. SELFCOMPACT matches or exceeds fixed-interval summarization at a fraction of the token cost, improving over a no-summarization baseline by up to 18.1 points on math and 5 to 9 points on agentic search, at 30 to 70 percent lower per-question cost.

The authors also describe a meta-cognitive gap: unprompted models cannot reliably tell when their own context is rotting, but a lightweight rubric closes this gap. They frame when to compact as a capability that scaffolds can supply without training.

What it means for model serving

For inference engineers serving open-weight models, the practical takeaway is that adaptive compaction can cut token spend on long agent runs without sacrificing accuracy. Because SELFCOMPACT requires no fine-tuning, it can be layered onto existing models as a scaffold, which matters for deployments where retraining is not an option. The rubric approach also suggests that structured prompts can substitute for model self-awareness in deciding when context has gone stale, a useful lever for controlling cost on agentic workloads.

The paper is authored by researchers from Apple and Johns Hopkins University. Related work from the same group includes rubric-based alignment for grounded answers and multi-environment agent training, both published in late 2026.