Transformer Architecture · 2024
Gated Delta Networks: Improving Mamba2 with Delta Rule (Gated DeltaNet)
Unifies gating (fast memory erasure) with the delta rule (targeted memory updates) into one hardware-parallel linear-attention layer, becoming the recurrent primitive behind later hybrid models.
Editorial record
Plain-language summary
Unifies gating (fast memory erasure) with the delta rule (targeted memory updates) into one hardware-parallel linear-attention layer, becoming the recurrent primitive behind later hybrid models.
Knowledge graph
Relationships
Descendants
Depends onEvidence: Strongly supported
Attention Is All You Need
Builds on the transformer architecture lineage in the archive
freshness sweep 2026
Source record
Provenance
- Record ID
- P-608
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2412.06464
- arXiv:2412.06464
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.