Open-Weight Model Families · 2025
Falcon-H1: A Family of Hybrid-Head Language Models
Runs Transformer attention and Mamba SSM heads in parallel within each block (tunable ratio), letting 0.5B-34B open-weight models rival much larger dense LLMs on long-context efficiency.
Editorial record
Plain-language summary
Runs Transformer attention and Mamba SSM heads in parallel within each block (tunable ratio), letting 0.5B-34B open-weight models rival much larger dense LLMs on long-context efficiency.
Knowledge graph
Relationships
Descendants
Depends onEvidence: Strongly supported
Attention Is All You Need
Builds on the open weight family lineage in the archive
freshness sweep 2026
Source record
Provenance
- Record ID
- P-634
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2507.22448
- arXiv:2507.22448
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.