Continual Learning & Memory · 2025

Cartridges: Lightweight and General-Purpose Long Context Representations via Self-Study

Sabri Eyuboglu, Ryan Ehrlich, Simran Arora, Neel Guha, Dylan Zinsley, Emily Liu, Will Tennien, Atri Rudra, James Zou, Azalia Mirhoseini, Christopher Ré

Primary source Repository

Introduced trained KV caches called Cartridges that replicate in-context learning functionality while consuming 38.6x less memory by offline training on text corpora through a Self-Study recipe.

Editorial record

Plain-language summary

Cartridges train a smaller KV cache offline on a corpus by generating synthetic conversations and using context distillation, then load this trained cache at inference time. The Self-Study recipe produces Cartridges that match ICL performance on challenging long-context benchmarks while enabling 26.4x higher throughput. Cartridges extend effective context length (e.g., 128k to 484k tokens on MTOB) and can be composed at inference without retraining, amortizing training cost across all queries referencing the same corpus.

Source record

Provenance

Record ID: P-666
Record created: 2026-08-12
Last reviewed: 2026-08-12
Record version: 1

https://arxiv.org/abs/2506.06266
arXiv:2506.06266

Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.