DeepSeek (V2/V3; MLA + efficient MoE + FP8)

Editorial record

Plain-language summary

DeepSeek-V3 is a mixture-of-experts model where only a small fraction of its total parameters activate per token, keeping compute per token low. It introduces multi-head latent attention, which compresses the attention key/value cache into a smaller latent representation to reduce memory during inference, and pairs it with an auxiliary-loss-free scheme for balancing which experts get used. Trained in FP8 mixed precision, it reached quality comparable to leading models while using far fewer GPU-hours, and the weights are downloadable, making it open-weight.