Colendi AIResearch
Menu

Transformer Architecture · 2025

Tensor Product Attention Is All You Need (TPA / T6)

Yifan Zhang, Yifeng Liu, Huizhuo Yuan, Zhen Qin, et al.

Primary source

Factorizes queries, keys, and values with contextual tensor decompositions to shrink the KV cache 10x-plus while matching or beating MHA/GQA/MLA and integrating cleanly with RoPE.

Editorial record

Plain-language summary

Factorizes queries, keys, and values with contextual tensor decompositions to shrink the KV cache 10x-plus while matching or beating MHA/GQA/MLA and integrating cleanly with RoPE.

Knowledge graph

Relationships

Descendants

  • Depends onEvidence: Strongly supported

    Attention Is All You Need

    Builds on the transformer architecture lineage in the archive

    freshness sweep 2026

Source record

Provenance

Record ID
P-617
Record created
2026-07-13
Last reviewed
2026-07-14
Record version
2

Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.