TD-Gammon

Editorial record

Plain-language summary

The program uses a neural network to estimate the expected outcome of a board position and trains it with TD(lambda), which adjusts predictions so that each position's estimated value moves toward the value of positions that follow it during play. By playing hundreds of thousands of games against itself it improved without expert-labeled positions or a hand-tuned evaluation function. It reached strong tournament-level play and even influenced human opening theory, providing an early demonstration that reinforcement learning combined with function approximation could master a large-state-space game.