Scaling Laws & Compute · 2023
Are Emergent Abilities of LLMs a Mirage?
This paper argues apparent emergent abilities are largely an artifact of discontinuous or nonlinear evaluation metrics rather than fundamental jumps in model capability, showing that switching to smooth, continuous metrics turns the sharp jumps into gradual, predictable improvement.
Editorial record
Plain-language summary
Schaeffer et al. show that metrics like exact-match or multiple-choice accuracy are harsh and nonlinear: a model's underlying per-token performance can improve smoothly while the reported score stays flat then spikes once the whole answer becomes correct. Re-scoring the same models and tasks with continuous or partial-credit metrics converts many reported emergent curves into steady, extrapolable trends. The work reframes much claimed emergence as a measurement choice, cautioning that metric selection, not new capability, can manufacture the appearance of a threshold.
Knowledge graph
Relationships
Descendants
ChallengesEvidence: Direct
Emergent Abilities of Large Language Models
Mirage shows emergence is partly a metric artifact
P-103 abstract
Source record
Provenance
- Record ID
- P-103
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2304.15004
- arXiv:2304.15004
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.