Where test-time compute spends more inference computation without changing weights, test-time training changes (some) weights at inference. TTT layers made the recurrent state of an RNN a small model whose update rule is a step of self-supervised learning; Google’s Titans scaled the idea into a neural long-term memory trained online with surprise-gated updates and a learned forgetting gate, followed by ATLAS and the Nested Learning/HOPE line of work.
The family is best understood as an upgrade of linear attention’s running state: instead of accumulating key-value associations by summation, the memory corrects itself with a gradient step (the delta rule), writing hard when surprised and decaying what the context no longer needs. It is one of the live candidate routes to continual learning in deployed models.