Q-learning

Editorial record

Plain-language summary

Watkins and Dayan defined a rule that updates an estimate of the long-run reward for each state-action pair (the Q-value) from observed transitions and rewards. Because updates use the best available next action rather than the action actually taken, the agent can learn the optimal policy while still exploring. They gave a convergence proof, giving reinforcement learning a model-free method with mathematical guarantees that underpins much later work.