Retrieval & Memory · 2021
FiD / RETRO / Atlas (scaling retrieval)
A line of retrieval-scaling methods (represented by RETRO, with Fusion-in-Decoder and Atlas) showed that conditioning generation on large retrieved text databases lets smaller models match much larger ones, removing the need to scale parameters to hold more knowledge.
Editorial record
Plain-language summary
These methods retrieve many relevant passages from a large corpus and fuse them into the generator: Fusion-in-Decoder encodes each passage separately and combines them in the decoder, RETRO cross-attends to chunk-level neighbors from a trillion-token database during pretraining, and Atlas jointly trains retriever and reader for few-shot knowledge tasks. Represented by RETRO, the family let models trade parameters for an external datastore, reaching strong language-modeling and question-answering results at lower model size while keeping the knowledge base updatable.
Knowledge graph
Relationships
Descendants
ExtendsEvidence: Direct
Retrieval-Augmented Generation (RAG)
FiD/RETRO/Atlas scale retrieval
P-323
Source record
Provenance
- Record ID
- P-323
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2112.04426
- arXiv:2112.04426
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.