Scaling Laws & Compute · 2020
The Scaling Hypothesis
Argued that the capabilities of large language models are largely a smooth function of scale, so continuing to scale compute and data should keep producing gains without new architectural ideas.
Editorial record
Plain-language summary
Written after GPT-3, the essay collects the evidence that loss and many capabilities improve predictably with size, and contends that the field had underrated how far pure scaling would go. It frames scaling as a hypothesis to be taken seriously rather than a curiosity. It is one of the defining statements of the scaling-era worldview and its stance is debated.
Knowledge graph
Relationships
Descendants
Depends onEvidence: Strongly supported
Scaling Laws for Neural Language Models
The scaling hypothesis draws on the empirical scaling laws
book
Source record
Provenance
- Record ID
- P-511
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.