Long Context & Efficient Sequences · 2025
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention (NSA)
Makes sparse attention trainable end-to-end and arithmetic-intensity-balanced for GPUs, cutting long-context train and decode cost without the quality loss of post-hoc sparsity.
Editorial record
Plain-language summary
Makes sparse attention trainable end-to-end and arithmetic-intensity-balanced for GPUs, cutting long-context train and decode cost without the quality loss of post-hoc sparsity.
Knowledge graph
Relationships
Descendants
Depends onEvidence: Strongly supported
Attention Is All You Need
Builds on the long context lineage in the archive
freshness sweep 2026
Source record
Provenance
- Record ID
- P-606
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2502.11089
- arXiv:2502.11089
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.