Fast Transformer Decoding: One Write-Head is All You Need (MQA)

Editorial record

Plain-language summary

In standard multi-head attention each head has its own key and value projections, so the per-step key/value cache that dominates incremental decoding is large and bandwidth-bound. MQA keeps multiple query heads but collapses to one shared key/value head, shrinking that cache by the number of heads and making token-by-token generation much faster. The trade-off is a small quality drop, but the decoding speedup made MQA a common choice for inference-heavy models and set up the later GQA compromise.