Colendi AIResearch
Menu

Research archive

The Intelligence Papers

A navigable record of the ideas, artifacts, and institutions that formed the modern intelligence stack—reviewed as evidence, not arranged as a reading list.

211reviewed records

22research domains

1843–2025year range

3 records

0012020Decoder-Only Language Models

Peer reviewed

Language Models are Few-Shot Learners (GPT-3)

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, et al.

GPT-3 trains a 175-billion-parameter Transformer on a filtered Common Crawl plus other corpora, keeping the next-token objective but scaling roughly 100x over GPT-2. Given a natural-language instruction and a handful of demonstrations in its context window, it performs translation, question answering, arithmetic, and other tasks without weight updates, with accuracy generally rising as more examples are shown. This removed the per-task fine-tuning and labeled-data requirement for many uses and made prompting the primary interface to large models.

UnknownDifficulty 6/10Verified
Read record Primary source
0022018Decoder-Only Language Models

Technical report

Improving Language Understanding by Generative Pre-Training (GPT)

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever

GPT-1 pre-trains a left-to-right Transformer on a large corpus of books to predict the next token, then adapts to each downstream task with a small supervised fine-tuning step and task-specific input formatting. This two-stage recipe removed the need to hand-design a separate model per task and to rely on scarce labeled data. It improved results on entailment, question answering, semantic similarity, and classification, establishing generative pre-training plus fine-tuning as a general NLP method.

UnknownDifficulty 5/10Verified
Read record Primary source
0032019Decoder-Only Language Models

Technical report

Language Models are Unsupervised Multitask Learners (GPT-2)

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever

GPT-2 trains a 1.5B-parameter Transformer on WebText, a large corpus scraped from outbound Reddit links, using the same next-token objective as GPT-1. The paper shows the model handles reading comprehension, translation, summarization, and question answering in a zero-shot setting when the task is phrased as text, without any task-specific training. This reframed NLP tasks as special cases of language modeling and gave early evidence that capability scales with model and data size.

UnknownDifficulty 5/10Verified
Read record Primary source