Long Context & Efficient Sequences · 2025
MoBA: Mixture of Block Attention for Long-Context LLMs
Applies MoE-style routing to attention so each query attends to a learned subset of key-value blocks, allowing a model to switch between full and sparse attention and serve long context in production (Kimi).
Editorial record
Plain-language summary
Applies MoE-style routing to attention so each query attends to a learned subset of key-value blocks, allowing a model to switch between full and sparse attention and serve long context in production (Kimi).
Knowledge graph
Relationships
Descendants
Depends onEvidence: Strongly supported
Attention Is All You Need
Builds on the long context lineage in the archive
freshness sweep 2026
Source record
Provenance
- Record ID
- P-607
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2502.13189
- arXiv:2502.13189
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.