Named by McCloskey and Cohen in 1989, catastrophic forgetting is the reason you cannot simply keep fine-tuning a deployed model on whatever it experiences. Distributed representations share weights across everything the network knows, so learning task B pulls on the same parameters task A depends on; French’s classic review points out that the very properties that make neural networks generalize are the root cause.
The standard mitigations each trade new-task performance for retention along a stability-plasticity frontier: regularization methods like elastic weight consolidation penalize movement of important weights, replay methods mix stored old examples into new training batches, and architectural methods give each task its own parameters (adapters, progressive networks, sparse memory slots). The effect is one of the main reasons LLM agents keep their memories in external stores rather than in their weights.