Neural Foundations · 2014
Neural Machine Translation by Jointly Learning to Align and Translate (Bahdanau attention)
Introduced an attention mechanism that lets the decoder compute a weighted combination of all encoder hidden states at each output step, removing the fixed-length bottleneck vector that capped encoder-decoder translation quality on long sentences.
Editorial record
Plain-language summary
Instead of forcing the encoder to compress a whole sentence into one vector, this model learns an alignment: for each output word the decoder scores and softly selects relevant source positions to build a context vector. This kept translation quality from degrading as sentences got longer and produced interpretable soft alignments between source and target words. The mechanism became the core building block that the Transformer later generalized.
Knowledge graph
Relationships
Antecedents
GeneralizesEvidence: Direct
Attention Is All You Need
Self-attention generalizes content-based soft alignment
P-001 cites Bahdanau 2014
Descendants
GeneralizesEvidence: Strongly supported
The Mathematics of Statistical MT (IBM alignment models)
Neural attention is soft learned alignment
A-007
ExtendsEvidence: Direct
Sequence to Sequence Learning with Neural Networks
Attention removes seq2seq bottleneck
A-007
Source record
Provenance
- Record ID
- A-007
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/1409.0473
- arXiv:1409.0473
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.