Research archive
A navigable record of the ideas, artifacts, and institutions that formed the modern intelligence stack—reviewed as evidence, not arranged as a reading list.
211reviewed records
22research domains
1843–2025year range
Technical report
Meta trained a family from 7B to 65B parameters exclusively on public datasets, pushing token counts well past Chinchilla-optimal points to favor cheaper inference. The 13B model matched or beat GPT-3 (175B) on many benchmarks and the 65B was competitive with Chinchilla and PaLM. Its release under a research license, and the subsequent weight availability, seeded a large open ecosystem of fine-tuned derivatives.
Technical report
The paper describes how the Llama 3 models (up to 405B parameters) were built: a large curated pretraining corpus, a standard dense transformer architecture scaled up, and a post-training pipeline of supervised fine-tuning and preference optimization for instruction following. It also covers extended context length, multilingual and code capabilities, and safety tuning. Because the weights are downloadable under a community license, it gave researchers and companies a strong base model to run and fine-tune locally, though the license and undisclosed training data mean it is open-weight rather than fully open source.
Technical report
DeepSeek-V3 is a mixture-of-experts model where only a small fraction of its total parameters activate per token, keeping compute per token low. It introduces multi-head latent attention, which compresses the attention key/value cache into a smaller latent representation to reduce memory during inference, and pairs it with an auxiliary-loss-free scheme for balancing which experts get used. Trained in FP8 mixed precision, it reached quality comparable to leading models while using far fewer GPU-hours, and the weights are downloadable, making it open-weight.
Technical report
Mistral 7B combines grouped-query attention (which shares key/value projections across query heads to cut memory and speed up inference) with sliding-window attention (which limits each token's attention to a fixed recent window so longer sequences stay cheap). With these changes it performed on par with or better than models roughly twice its size on common benchmarks. Released with downloadable weights under a permissive Apache 2.0 license, it became a practical base for local deployment and fine-tuning.
Technical report
Qwen2.5 scales up the pretraining data to roughly 18 trillion tokens and refines the post-training pipeline (supervised fine-tuning plus reinforcement learning from preferences) across a range of model sizes from small to large. It targets improvements in instruction following, structured output, mathematics, and coding, and supports long context windows and many languages. The base and instruction-tuned weights are downloadable, giving practitioners a broad menu of sizes to fit different hardware budgets; most sizes are released under a permissive license while some larger ones use a more restrictive one, so it is open-weight.
Technical report
Gemma, Phi (small models trained on curated/synthetic 'textbook-quality' data), Falcon, Command R (retrieval and tool-use oriented), Yi, and GLM each shipped capable open-weight models with differing licenses and disclosure. OLMo is fully open: alongside weights it releases the Dolma training corpus, the training and evaluation code, and intermediate checkpoints, making it a scientific artifact rather than only a usable model. The distinction matters because most 'open' models share only weights, whereas OLMo lets researchers study and reproduce how a given capability arose during training.
Peer reviewed
GLM masks spans of text and trains the model to regenerate them autoregressively while attending bidirectionally to the surrounding context, giving one model both the understanding strengths of masked encoders like BERT and the generation ability of decoder-only models. The team scaled the recipe to GLM-130B, a 130-billion-parameter bilingual model released with open weights and INT4 inference support so it could run on modest hardware. It anchored the GLM/ChatGLM family from Zhipu AI and Tsinghua and became one of the more capable early open bilingual foundation models.
Technical report
From GLM-4.5 (a 355B-parameter sparse mixture-of-experts model with 32B active, July 2025) through the GLM-5 family and the GLM-5.2 flagship (a roughly 750B-parameter MoE with sparse attention and a million-token context, 2026), Zhipu AI released the weights on Hugging Face under MIT, with FP8 variants for cheaper serving. The series targets reasoning and agentic software engineering and is also served through the z.ai API, but the checkpoints are genuinely public and self-hostable. It is among the most capable open-weight lines outside the Llama, Qwen, and DeepSeek families. (GLM-5.x release dates are approximate, pinned from repository timestamps rather than day-precise announcements.)
Technical report
BLOOM was trained by the BigScience workshop on the ROOTS corpus spanning 46 natural and 13 programming languages, with the model, code, and data documentation released openly under a responsible-AI licence. It brought many languages underserved by English-centric models into a large open model and made the full training process transparent. It was an early proof that open, multi-institution collaboration could build models at frontier scale.