Open-Weight Model Families · 2024
Open Ecosystem (Gemma/Phi/OLMo/Falcon/Command R/Yi/GLM/InternLM)
This open ecosystem produced strong sub-frontier base models under varied openness levels, with OLMo going furthest by releasing not just weights but the full training data, code, logs, and checkpoints so the entire training process is reproducible.
Editorial record
Plain-language summary
Gemma, Phi (small models trained on curated/synthetic 'textbook-quality' data), Falcon, Command R (retrieval and tool-use oriented), Yi, and GLM each shipped capable open-weight models with differing licenses and disclosure. OLMo is fully open: alongside weights it releases the Dolma training corpus, the training and evaluation code, and intermediate checkpoints, making it a scientific artifact rather than only a usable model. The distinction matters because most 'open' models share only weights, whereas OLMo lets researchers study and reproduce how a given capability arose during training.
Knowledge graph
Relationships
Descendants
ExtendsEvidence: Strongly supported
LLaMA: Open and Efficient Foundation Language Models
Broad open ecosystem follows LLaMA
P-365
EnablesEvidence: Strongly supported
Distilling the Knowledge in a Neural Network
Open small models such as Gemma distil from larger frontier teachers
model reports
Source record
Provenance
- Record ID
- P-365
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2402.00838
- arXiv:2402.00838
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.