Reasoning & Test-Time Compute · 2023
Verifiers & Process Supervision (GSM8K / Let's Verify Step by Step)
It compared rewarding only the final answer (outcome supervision) against rewarding each intermediate reasoning step (process supervision) and showed that step-level feedback trains a more reliable verifier that selects correct solutions to hard math problems more often.
Editorial record
Plain-language summary
The authors had humans label the correctness of every step in a model's chain of reasoning, then trained a verifier to score solutions step by step. Ranking many candidate solutions by this process-supervised verifier picked correct answers more often than a verifier trained only on whether the final answer was right. They also released PRM800K, a large dataset of step-level human labels, making process-based reward modeling reproducible for reasoning tasks.
Knowledge graph
Relationships
Antecedents
Depends onEvidence: Direct
DeepSeek-R1: Incentivizing Reasoning via RL (RLVR)
RLVR uses verifiers/process rewards
P-224
Source record
Provenance
- Record ID
- P-223
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2305.20050
- arXiv:2305.20050
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.