Open-Weight Model Families · 2023
LLaMA: Open and Efficient Foundation Language Models
Showed that training smaller models on far more publicly available tokens produces compute-efficient foundation models, removing the assumption that strong performance required proprietary data or the largest parameter counts.
Editorial record
Plain-language summary
Meta trained a family from 7B to 65B parameters exclusively on public datasets, pushing token counts well past Chinchilla-optimal points to favor cheaper inference. The 13B model matched or beat GPT-3 (175B) on many benchmarks and the 65B was competitive with Chinchilla and PaLM. Its release under a research license, and the subsequent weight availability, seeded a large open ecosystem of fine-tuned derivatives.
Knowledge graph
Relationships
Antecedents
ExtendsEvidence: Direct
Llama 2 & Llama 3
Llama 2/3 extend LLaMA
P-361
ExtendsEvidence: Strongly supported
Mistral 7B / Mixtral
Mistral builds on the open block
P-362
ExtendsEvidence: Strongly supported
Qwen (Qwen/Qwen2/Qwen2.5)
Qwen builds on the open paradigm
P-363
ExtendsEvidence: Strongly supported
DeepSeek (V2/V3; MLA + efficient MoE + FP8)
DeepSeek builds on the open paradigm
P-364
ExtendsEvidence: Strongly supported
Open Ecosystem (Gemma/Phi/OLMo/Falcon/Command R/Yi/GLM/InternLM)
Broad open ecosystem follows LLaMA
P-365
Parallel developmentEvidence: Strongly supported
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BLOOM and LLaMA are parallel large open language models
open ecosystem
Descendants
EnablesEvidence: Direct
Training Compute-Optimal Large Language Models (Chinchilla)
Chinchilla-informed over-training shapes LLaMA
P-360
Source record
Provenance
- Record ID
- P-360
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2302.13971
- arXiv:2302.13971
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.