Inference & Serving · 2025
Defeating Nondeterminism in LLM Inference
Traces run-to-run nondeterminism in LLM serving to batch-size-dependent kernel reductions rather than float non-associativity, and gives batch-invariant RMSNorm/matmul/attention kernels that make inference bitwise-reproducible.
Editorial record
Plain-language summary
Traces run-to-run nondeterminism in LLM serving to batch-size-dependent kernel reductions rather than float non-associativity, and gives batch-invariant RMSNorm/matmul/attention kernels that make inference bitwise-reproducible.
Knowledge graph
Relationships
Descendants
Depends onEvidence: Strongly supported
Attention Is All You Need
Concerns deterministic inference of Transformer models
TML
Source record
Provenance
- Record ID
- P-655
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.