Colendi AIResearch
Menu

Research archive

The Intelligence Papers

A navigable record of the ideas, artifacts, and institutions that formed the modern intelligence stack—reviewed as evidence, not arranged as a reading list.

211reviewed records

22research domains

1843–2025year range

2 records

0012017Continual Learning & Memory

Peer reviewed

Overcoming Catastrophic Forgetting in Neural Networks

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Raia Hadsell, Demis Hassabis

The method estimates how sensitive prior-task performance is to each weight using the Fisher information, then adds a quadratic penalty anchoring the important weights while leaving the rest free to adapt. A single network could learn a sequence of tasks and retain earlier ones where ordinary training would overwrite them. It turned catastrophic forgetting from a vague failure into a measurable, tractable problem and anchored the continual-learning literature.

UnknownDifficulty 6/10Verified
Read record Primary source
0022025Continual Learning & Memory

Preprint

Titans: Learning to Memorize at Test Time

Ali Behrouz, Peilin Zhong, Vahab Mirrokni

Titans pairs short-term attention with a deep neural memory updated online by a surprise-based gradient signal, so salient new information is written into the memory as the model reads, with selective forgetting. This yields effective context well beyond the attention window while keeping inference cost bounded. It is a leading example of the test-time-learning branch, where memory lives in parameters updated at inference rather than only in the context window.

UnknownDifficulty 6/10Verified
Read record Primary source