Speculative Decoding / Medusa / EAGLE

Editorial record

Plain-language summary

Autoregressive generation is slow because each token requires a full forward pass of the large model, run one at a time. Speculative decoding runs a cheap draft model to guess a short run of upcoming tokens, then the large model checks all of them in a single parallel pass and accepts the longest correct prefix, resampling at the first mismatch. A modified acceptance rule guarantees the result matches what the large model would have produced on its own, so it speeds up inference (often 2-3x) with no loss in output quality.