Retrieval & Memory · 2019
Contrastive Text Embeddings (Sentence-BERT/SimCSE/E5/BGE)
Introduced Sentence-BERT, which fine-tunes BERT in a siamese setup to produce sentence embeddings comparable with cosine similarity, removing the need to run BERT on every sentence pair and making large-scale semantic similarity and search practical.
Editorial record
Plain-language summary
Standard BERT compares two sentences by feeding them together, so finding the best match in a large collection needs a separate forward pass per pair, which is far too slow. SBERT instead passes each sentence through a shared BERT with pooling to get one fixed vector per sentence, trained on natural-language-inference and similarity data so that similar meanings give nearby vectors. This lets you embed a corpus once and then compare sentences with cheap vector operations, enabling clustering, semantic search, and duplicate detection at scale.
Knowledge graph
Relationships
Antecedents
Converts into infrastructureEvidence: Strongly supported
Retrieval-Augmented Generation (RAG)
Embeddings power production RAG
P-325
Descendants
Applies toEvidence: Direct
BERT: Pre-training of Deep Bidirectional Transformers
Sentence-BERT adapts BERT for embeddings
P-325
Source record
Provenance
- Record ID
- P-325
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/1908.10084
- arXiv:1908.10084
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.