Multimodality · 2023
BLIP / BLIP-2 (Q-Former)
BLIP-2 introduced a lightweight Querying Transformer (Q-Former) that connects a frozen image encoder to a frozen large language model, removing the need to train either backbone to build a vision-language model.
Editorial record
Plain-language summary
BLIP-2 keeps a pretrained image encoder and a pretrained LLM frozen and trains only a small Q-Former, a set of learnable query tokens that extract the visual features most relevant to text and feed them into the language model. This two-stage bridging cuts the trainable parameters and compute needed for vision-language pretraining by orders of magnitude, letting new image encoders and LLMs be combined into captioning and visual question-answering systems without full end-to-end retraining.
Knowledge graph
Relationships
Antecedents
ExtendsEvidence: Strongly supported
Open Production VLMs (Qwen-VL/InternVL/PaLI/Kosmos)
Q-Former pattern feeds production VLMs
P-306
Descendants
EnablesEvidence: Direct
CLIP: Learning Transferable Visual Models from NL Supervision
CLIP encoder feeds BLIP-2
P-304
Source record
Provenance
- Record ID
- P-304
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2301.12597
- arXiv:2301.12597
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.