Effective Approaches to Attention-based NMT (Luong attention)

Editorial record

Plain-language summary

The paper lets a translation decoder, at each output step, compute a weighted average of encoder hidden states where the weights come from a similarity score between the current decoder state and each source state, then combines that context with the decoder state to predict the next word. It compared scoring functions (dot product, general, concat) and introduced a 'local' attention that only looks at a small window of source positions to save computation. These variants improved translation quality and clarified the design space, and the dot-product scoring it favored became standard in later attention and Transformer work.