Toolformer: LMs Can Teach Themselves to Use Tools

Editorial record

Plain-language summary

The model samples candidate API calls (calculator, search, calendar, question-answering, translation) inside ordinary text, executes them, and keeps only the calls whose returned results lower the perplexity of the following tokens, producing a training set it then fine-tunes on. As a result a 6.7B-parameter model learns to invoke tools where they help and gains on arithmetic, factual QA, and temporal tasks without degrading its core language modeling. The mechanism showed tool use can be bootstrapped from the model's own predictions rather than curated examples.