Titans: Learning to Memorize at Test Time

Editorial record

Plain-language summary

Titans pairs short-term attention with a deep neural memory updated online by a surprise-based gradient signal, so salient new information is written into the memory as the model reads, with selective forgetting. This yields effective context well beyond the attention window while keeping inference cost bounded. It is a leading example of the test-time-learning branch, where memory lives in parameters updated at inference rather than only in the context window.