Open-Weight Model Families · 2021
GLM: General Language Model Pretraining with Autoregressive Blank Infilling / GLM-130B
Introduced the GLM pretraining objective — autoregressive blank infilling — which unifies bidirectional context and left-to-right generation in a single model, and scaled it into GLM-130B, an openly released English–Chinese bilingual foundation model.
Editorial record
Plain-language summary
GLM masks spans of text and trains the model to regenerate them autoregressively while attending bidirectionally to the surrounding context, giving one model both the understanding strengths of masked encoders like BERT and the generation ability of decoder-only models. The team scaled the recipe to GLM-130B, a 130-billion-parameter bilingual model released with open weights and INT4 inference support so it could run on modest hardware. It anchored the GLM/ChatGLM family from Zhipu AI and Tsinghua and became one of the more capable early open bilingual foundation models.
Knowledge graph
Relationships
Antecedents
Parallel developmentEvidence: Probable
Qwen (Qwen/Qwen2/Qwen2.5)
GLM and Qwen are parallel open-weight bilingual model families
model families
Conceptual ancestorEvidence: Strongly supported
GLM-4.5 to GLM-5.2 (Zhipu AI / Z.ai open-weight MoE series)
The modern open-weight GLM series continues the GLM lineage
zai-org HF
Descendants
Depends onEvidence: Strongly supported
Attention Is All You Need
GLM is built on the Transformer architecture
GLM 2103.10360
CombinesEvidence: Strongly supported
BERT: Pre-training of Deep Bidirectional Transformers
GLM blank-infilling incorporates BERT-style bidirectional masked prediction
GLM 2103.10360
CombinesEvidence: Strongly supported
Improving Language Understanding by Generative Pre-Training (GPT)
GLM unifies GPT-style autoregressive generation with bidirectional context
GLM 2103.10360
Parallel developmentEvidence: Strongly supported
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5)
GLM and T5 are parallel unified pretraining-objective approaches
GLM 2103.10360
Source record
Provenance
- Record ID
- P-366
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2103.10360
- arXiv:2103.10360
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.