CLIP: Learning Transferable Visual Models from NL Supervision

Editorial record

Plain-language summary

Two encoders (one for images, one for text) are trained so that a photo and its matching caption land close together in a shared embedding space while mismatched pairs are pushed apart. To classify a new image with no task-specific training, you write candidate labels as text prompts and pick the one whose embedding is closest to the image. Because the labels are open-ended text rather than a fixed category list, one model transfers to many recognition tasks and became a reusable component for retrieval and for later image-generation and vision-language systems.