Long Short-Term Memory (LSTM)

Editorial record

Plain-language summary

Each LSTM unit keeps a memory cell whose value is protected by gates that control when new information is written in, retained, or read out, so the cell can hold information across many time steps. Because the cell passes its state forward with near-constant behavior when the gates allow, gradients do not shrink away over long sequences the way they do in plain recurrent networks. This let recurrent networks learn dependencies spanning hundreds of steps and made them effective for sequence tasks such as speech and language modeling.