Open-Weight Model Families · 2024
Llama 2 & Llama 3
A detailed engineering report on Meta's Llama family of open-weight dense transformer models, documenting the data, scaling, and post-training recipe that made a widely downloadable model competitive with the strongest closed models of its time.
Editorial record
Plain-language summary
The paper describes how the Llama 3 models (up to 405B parameters) were built: a large curated pretraining corpus, a standard dense transformer architecture scaled up, and a post-training pipeline of supervised fine-tuning and preference optimization for instruction following. It also covers extended context length, multilingual and code capabilities, and safety tuning. Because the weights are downloadable under a community license, it gave researchers and companies a strong base model to run and fine-tune locally, though the license and undisclosed training data mean it is open-weight rather than fully open source.
Knowledge graph
Relationships
Descendants
ExtendsEvidence: Direct
LLaMA: Open and Efficient Foundation Language Models
Llama 2/3 extend LLaMA
P-361
Source record
Provenance
- Record ID
- P-361
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2407.21783
- arXiv:2407.21783
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.