Alignment & Preference Learning · 2022
Finetuned Language Models Are Zero-Shot Learners (FLAN) / Self-Instruct
It showed that fine-tuning a language model on a broad collection of tasks phrased as natural-language instructions makes the model follow instructions for unseen tasks it was never trained on, removing the need for task-specific examples or prompt engineering to steer a model.
Editorial record
Plain-language summary
The authors took a pretrained model and fine-tuned it on many existing NLP datasets that were each rewritten as instructions (for example, 'Is this review positive or negative?'). After this instruction tuning, the model could handle new kinds of tasks it had not seen during training, just from reading the instruction. This established instruction tuning as a general method for making base models usable without few-shot prompting, and improved zero-shot performance across a range of benchmarks.
Knowledge graph
Relationships
Antecedents
Depends onEvidence: Strongly supported
InstructGPT: Training LMs to Follow Instructions with Human Feedback
SFT/instruction data is stage 1 of RLHF
P-202
Source record
Provenance
- Record ID
- P-203
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2109.01652
- arXiv:2109.01652
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.