WebGPT: Browser-assisted Question-answering with Human Feedback

Editorial record

Plain-language summary

WebGPT gives GPT-3 a text-based browser and trains it with human comparisons to search, navigate, and quote evidence, then optimizes against a learned reward model. The result answers long-form questions with references that raters prefer, at the cost of sometimes over-trusting sources. It was an early demonstration of tool use and retrieval-augmented answering with RLHF.