Finding Structure in Time (Elman RNN)

Editorial record

Plain-language summary

The network copies its hidden-layer state at each step into a 'context' layer that is fed back in alongside the next input, so the hidden units learn representations that depend on prior elements of the sequence. Trained on tasks like predicting the next word or discovering word boundaries in a stream, it developed internal representations that reflected grammatical categories and structure without being told them. This showed that temporal structure could be handled by recurrence rather than by explicit time-delay windows, and the architecture became a foundation for later recurrent networks.