Transformer Architecture · 2024
Better & Faster Large Language Models via Multi-token Prediction
Training with multiple parallel prediction heads improves sample efficiency and enables self-speculative decoding, and became the multi-token-prediction objective adopted by DeepSeek-V3 and later models.
Editorial record
Plain-language summary
Training with multiple parallel prediction heads improves sample efficiency and enables self-speculative decoding, and became the multi-token-prediction objective adopted by DeepSeek-V3 and later models.
Knowledge graph
Relationships
Descendants
Depends onEvidence: Strongly supported
Attention Is All You Need
Builds on the transformer architecture lineage in the archive
freshness sweep 2026
Source record
Provenance
- Record ID
- P-614
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2404.19737
- arXiv:2404.19737
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.