Long Context & Efficient Sequences · 2023
Lost in the Middle / Long-Context Reliability (RULER)
An empirical study showing that long-context models retrieve information reliably only when it sits near the start or end of the input and degrade sharply for content in the middle, exposing that a large context window does not guarantee uniform use of it (with RULER later giving a systematic length-vs-reliability probe).
Editorial record
Plain-language summary
The work tested models on multi-document QA and key-value retrieval while varying where the relevant information was placed in a long input. Accuracy followed a U-shaped curve: high when the needed fact was at the beginning or end, substantially lower when buried in the middle, even for models nominally supporting the full length. This demonstrated that advertised context length overstates usable context, motivating position-robustness work and synthetic stress benchmarks like RULER that measure at what effective length a model still retrieves and reasons reliably.
Source record
Provenance
- Record ID
- P-427
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2307.03172
- arXiv:2307.03172
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.