Mamba / Mamba-2 (Selective State Spaces)

Editorial record

Plain-language summary

The authors made structured state-space model parameters input-dependent so the model can selectively remember or forget information along a sequence, then designed a hardware-aware parallel scan that computes the recurrence efficiently on GPUs without materializing the full state. Mamba runs in linear time with constant memory per step at inference and handles very long sequences, while matching or exceeding Transformers of similar size on language, audio, and genomics. It offered an attention-free architecture with higher inference throughput for long-context modeling.