Colendi AIResearch
Menu

Research archive

The Intelligence Papers

A navigable record of the ideas, artifacts, and institutions that formed the modern intelligence stack—reviewed as evidence, not arranged as a reading list.

211reviewed records

22research domains

1843–2025year range

8 records

0012020Retrieval & Memory

Peer reviewed

Retrieval-Augmented Generation (RAG)

Patrick Lewis, Ethan Perez, Aleksandra Piktus, et al.

RAG pairs a trained generator with a retriever that fetches relevant passages from an external corpus (a Wikipedia index searched with dense vectors), then conditions the output on those passages. Both the retriever's query encoder and the generator are trained together, and retrieved documents can be swapped or updated without retraining the model. This lets a model cite and use up-to-date or domain-specific knowledge, reducing fabricated answers on knowledge-intensive tasks like open-domain question answering.

UnknownDifficulty 5/10Verified
Read record Primary source
0022020Retrieval & Memory

Peer reviewed

Dense Passage Retrieval (DPR)

Vladimir Karpukhin, Barlas Oguz, Sewon Min, et al.

Two BERT encoders map questions and passages into a shared vector space, trained so a question sits near passages that answer it, using in-batch negatives to make training efficient. At query time the question is embedded once and nearest passages are found by fast vector similarity search over a precomputed index. Replacing term-overlap retrieval with learned semantic matching improved retrieval and downstream answer accuracy, and established the dense-retriever design widely used in retrieval-augmented pipelines.

UnknownDifficulty 5/10Verified
Read record Primary source
0032019Retrieval & Memory

Peer reviewed

Contrastive Text Embeddings (Sentence-BERT/SimCSE/E5/BGE)

Nils Reimers, Iryna Gurevych, Tianyu Gao, Danqi Chen, Liang Wang, Furu Wei, Shitao Xiao, Zheng Liu

Standard BERT compares two sentences by feeding them together, so finding the best match in a large collection needs a separate forward pass per pair, which is far too slow. SBERT instead passes each sentence through a shared BERT with pooling to get one fixed vector per sentence, trained on natural-language-inference and similarity data so that similar meanings give nearby vectors. This lets you embed a corpus once and then compare sentences with cheap vector operations, enabling clustering, semantic search, and duplicate detection at scale.

UnknownDifficulty 4/10Verified
Read record Primary source
0042020Retrieval & Memory

Peer reviewed

REALM: Retrieval-Augmented LM Pre-Training

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, Ming-Wei Chang

REALM augments masked language-model pretraining with a learned retriever that fetches relevant documents from a corpus and conditions predictions on them, backpropagating the language-modeling signal through the retrieval step so the retriever learns which passages help. This let a relatively small model consult an external, updatable knowledge store instead of memorizing facts in its weights, improving open-domain question answering and making the knowledge source inspectable and swappable.

UnknownDifficulty 6/10Verified
Read record Primary source
0052021Retrieval & Memory

Peer reviewed

FiD / RETRO / Atlas (scaling retrieval)

Gautier Izacard, Edouard Grave, Sebastian Borgeaud, Jack W. Rae, Laurent Sifre

These methods retrieve many relevant passages from a large corpus and fuse them into the generator: Fusion-in-Decoder encodes each passage separately and combines them in the decoder, RETRO cross-attends to chunk-level neighbors from a trillion-token database during pretraining, and Atlas jointly trains retriever and reader for few-shot knowledge tasks. Represented by RETRO, the family let models trade parameters for an external datastore, reaching strong language-modeling and question-answering results at lower model size while keeping the knowledge base updatable.

UnknownDifficulty 6/10Verified
Read record Primary source
0062020Retrieval & Memory

Peer reviewed

ColBERT / Late Interaction / Rerankers / kNN-LM

Omar Khattab, Matei Zaharia, Urvashi Khandelwal, Mike Lewis, Dan Jurafsky

ColBERT represents each query and document as a bag of contextual token embeddings and computes relevance by summing, for each query token, its maximum similarity to any document token, so most document computation is precomputed offline and only lightweight matching happens at query time. This late-interaction design, alongside related rerankers and nearest-neighbor language models, delivered near cross-encoder ranking quality at a scale and latency close to bag-of-vectors search, making high-quality dense retrieval practical over large collections.

UnknownDifficulty 5/10Verified
Read record Primary source
0072022Retrieval & Memory

Peer reviewed

Knowledge Editing (ROME / MEMIT)

Kevin Meng, David Bau, Alex Andonian, Yonatan Belinkov

ROME uses causal tracing to identify the mid-layer feed-forward modules that store a given factual association, then applies a rank-one weight edit that rewrites that fact while leaving unrelated knowledge intact; MEMIT extends the same mechanism to insert thousands of edits at once. This gave a targeted, low-cost way to update or correct specific facts in a trained model and provided evidence for where and how factual knowledge is stored inside transformers.

UnknownDifficulty 6/10Verified
Read record Primary source
0082024Retrieval & Memory

Technical report

Graph & Structured Retrieval (GraphRAG)

Darren Edge, Ha Trinh, et al.

GraphRAG uses a language model to extract entities and relationships into a graph, clusters the graph into communities, and precomputes summaries for each community; queries are then answered by combining these community summaries rather than only retrieving isolated text chunks. This structured approach lets the system answer broad, whole-corpus sensemaking questions (such as overarching themes) that flat vector retrieval handles poorly, while keeping evidence traceable to source entities and relations.

UnknownDifficulty 5/10Verified
Read record Primary source