Glossary Entry

Context Rot

The empirical degradation of an LLM's recall and reasoning as its context window fills, long before the advertised token limit is reached.

LLMs Metrics

Seed source: Chroma Research 2025

Benchmarks established the effect in stages: RULER (2024) showed claimed context lengths far exceed usable ones, NoLiMa (2025) showed the drop is much worse when retrieval requires semantic association rather than literal string matching (GPT-4o falls from 99.3% to 69.7% within its window), and Chroma’s 2025 report found the pattern across 18 frontier models even on trivially simple tasks. Anthropic’s context-engineering guidance states it directly: as tokens in the window increase, the model’s ability to accurately recall information from that context decreases.

Context rot is the reason “just use a longer context” is not a memory strategy, and it motivates the standard agent mitigations: compaction, structured note-taking outside the window, retrieval of only what the current step needs, and sub-agents that return condensed summaries.